-- Logs begin at Mon 2022-05-23 12:24:31 UTC, end at Mon 2022-05-23 14:26:43 UTC. -- May 23 13:08:11.882905 dv-cinder-825492-14-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 23 13:08:11.892501 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon May 23 13:08:11 2022] *** May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: nodename: dv-cinder-825492-14-iscsi May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: machine: x86_64 May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: clock source: unix May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: pcre jit disabled May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: detected number of CPU cores: 4 May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: current working directory: / May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: detected binary path: /usr/bin/uwsgi-core May 23 13:08:11.895495 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: your processes number limit is 47680 May 23 13:08:11.896141 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: your memory page size is 4096 bytes May 23 13:08:11.896141 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: detected max file descriptor number: 2048 May 23 13:08:11.896141 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: lock engine: pthread robust mutexes May 23 13:08:11.896141 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: thunder lock: enabled May 23 13:08:11.896141 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 23 13:08:11.896141 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 23 13:08:11.941031 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: Python main interpreter initialized at 0x5602ab4329c0 May 23 13:08:11.941031 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: python threads support enabled May 23 13:08:11.941031 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: your server socket listen backlog is limited to 100 connections May 23 13:08:11.941031 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: your mercy for graceful operations on workers is 90 seconds May 23 13:08:11.941398 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: mapped 403077 bytes (393 KB) for 2 cores May 23 13:08:11.941476 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: *** Operational MODE: preforking *** May 23 13:08:11.941727 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: *** uWSGI is running in multiple interpreter mode *** May 23 13:08:11.941727 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: spawned uWSGI master process (pid: 80383) May 23 13:08:11.941931 dv-cinder-825492-14-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 23 13:08:11.943339 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: spawned uWSGI worker 1 (pid: 80395, cores: 1) May 23 13:08:11.943339 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: spawned uWSGI worker 2 (pid: 80396, cores: 1) May 23 13:08:11.943339 dv-cinder-825492-14-iscsi devstack@keystone.service[80383]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 23 13:08:17.657860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 13:08:17.657860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 23 13:08:17.658911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 13:08:17.658911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: from cryptography.utils import int_from_bytes May 23 13:08:17.694185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 13:08:17.694185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: from cryptography.utils import int_from_bytes May 23 13:08:17.694783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 23 13:08:17.694783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: from cryptography.utils import int_from_bytes May 23 13:08:17.709366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 13:08:17.709556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 13:08:17.709749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] command line args: [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 13:08:17.710034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 13:08:17.710191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ================================================================================ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 13:08:17.710441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.710684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_dir = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.710869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.711047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] config_source = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.711225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] debug = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.711421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.711783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.711961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.712211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.712459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.712729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.712878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_config_append = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.713051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.713225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_dir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.713387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.713600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.714386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.715063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.715063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.715287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.715402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.715649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.715893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.716127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.716358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.716609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.716842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.717084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.717324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.717606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.718084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.718256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] publish_errors = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.718483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.718710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.718884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.719126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.719356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.719588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] standard_threads = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.719815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.720054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.720227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.720399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_journal = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.720584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_json = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.720762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_stderr = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.720933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] use_syslog = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.721159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.721412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.721649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.721903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.722138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.722382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.722616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.722859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.723091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.723339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.723610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.723810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.724040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.724282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.724514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.724786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.725029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.725261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.725509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.725761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.726001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.726234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.726483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.726739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.726987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.727231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.727457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.727683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.727909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.728139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.728383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.728646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.728874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.729100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.729340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.730261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.730500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.730765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.730974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.731208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.731439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.731666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.731894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.732122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.732350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.732606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.737986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.738330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.738330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.738520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.738854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.738993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.739228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.739461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.739704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.739943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.740207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.740460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.740720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.740958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.741188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.741420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.741652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.741911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.742151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.742390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.742626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.742866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.743102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.743336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.743569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.743727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 23 13:08:17.743903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.744001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 23 13:08:17.744079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.744151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] command line args: [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 23 13:08:17.744220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.744355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 23 13:08:17.744441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.744522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ================================================================================ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 23 13:08:17.744681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.744807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] admin_token = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.744919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.745080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_dir = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.745575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.745684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.745763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.745879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] config_source = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.745879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] debug = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.746062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.746167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.746293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.746391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.746519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.746627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.746725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.746837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.746937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.external = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.747045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.747163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.747240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.747240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_config_append = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.747432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.747506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.747631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.747707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_dir = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.747778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.password = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.747901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.747978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] auth.token = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.748049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.748173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.748257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.748341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.748499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.748593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.748697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.748776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.748901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.748979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.749052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.749177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.749249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.749321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.749443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.749520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.749664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.749755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.749901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.749987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.750141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.750263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.750375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.750470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.750621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.750701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.750866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.750947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.751293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.751380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.751456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.751820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.751903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.70/identity {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.751903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] publish_errors = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.752048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.752123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.752193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.752313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.752389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.752533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.752634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.752719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.752871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.752994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.753128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.753217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] standard_threads = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.753363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.753444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.753599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.753678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.753778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.753864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.753939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_journal = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.754078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.754166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_json = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.754278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_stderr = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.754363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.754445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] use_syslog = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.754541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.754673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 23 13:08:17.754776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.754932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.755918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.756974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.757921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.758999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.759973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.760051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.760217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.760299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.760474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.760558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.761111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.761265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.765996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.766993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.767947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.768958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.769936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.770933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.771929 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.772956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.773979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774633 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.774945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.775962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.776113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.776193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.776928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.777519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.777519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.777722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.777800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.777873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.777960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.778979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.779924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.external = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.780966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.password = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] auth.token = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.781920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.782848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783226 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.783922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.784955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.785857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.786950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.787990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.788981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.789945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.790944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.791897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.792981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.793082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.793159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.793232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.794972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.795998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.796931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.797061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.797208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.797320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80395) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 13:08:17.797472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.797701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.797949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.798197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.798432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.798679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.798927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.799159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.799396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.799631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.799869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.800178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.800425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.800688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.800930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.801181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.801429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.801661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.803084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.803409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.803665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.803905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.804143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.804383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.804639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.804884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.805145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.805380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.805628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.805875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.806121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.806359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.806589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.806826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.807061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.807298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.807765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.808034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.808290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.808545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.808857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.809111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.809346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.809597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.809837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.810076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.810319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.810556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.810803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.811053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.811292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.811550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.811784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.812034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.812272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.812509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.812772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.813021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.813262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.813586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.813830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.814071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.814322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.814571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.814881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.815134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.815387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.815624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.815872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.816110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.816348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.816614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.816854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.817089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.817340 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.817588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.817833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.818080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.818328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.818575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.818822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.819071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.819305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.819551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.819796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.820076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.820339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.820798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.821036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.821287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.821550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.821785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.828837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.829959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.830573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.830573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.830573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.830573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.830817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.830895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.830967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.831039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.831213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.831869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.832121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.832359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.832626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.832867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.833116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.833350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.833595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.833827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.834068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.834304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 23 13:08:17.834473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80396) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 23 13:08:17.838676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.860923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.861150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.861376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.868006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.869172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.872011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.891050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.891280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.930197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.931535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.982890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.983124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.984150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:17.984388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.053504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.053757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.053827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.053934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.054785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.084536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.084818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.084896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.084976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.085934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.098021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.098383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.111934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.112017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.112145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.112229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.112355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80395) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.112437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.112586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80396) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 23 13:08:18.130289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.130437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.130523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 13:08:18.130629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 13:08:18.130826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.130962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.131062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 13:08:18.131142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 23 13:08:18.131293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.131413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.131554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.131670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.131819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.131903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.132076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.132157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.143674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.148498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.153203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.155067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.156466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.156764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.157047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.157364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.157642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.157895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.158155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.158391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.158532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.158926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.159051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.159211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.159304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.159493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.159575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.159776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.160152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.160424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.160723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.160996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.161268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.161537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.187468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.189247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.189353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.189524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.191044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.191325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.194769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.195048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.195326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.196677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.196960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.197241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.203471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.203747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.205154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.205441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.208216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.208668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.208958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.209238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.209507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.216208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.216729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.229186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.230005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.231224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.231507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.234783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.236217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.238489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.238780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.239915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.240212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.243831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.247803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.248063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.251462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.251852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.252145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.252432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.252741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.253024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.253304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.255269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.255579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.255864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.256156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.256437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.256757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.272384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.272694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.272960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.273220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.275989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.276300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.276660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.276927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.281237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.285395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.285395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.285531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.286465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.286743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.286918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.287014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.287203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.287286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.287548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.290291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.290576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.290858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.291195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.291478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.301771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.302047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.302323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.302605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.302881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.303151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.303427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.305615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.305899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.306179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.306468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.306751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.307027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.307311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.322121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.325501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.326046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.328833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.329357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.330418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.332012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.332297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.332602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.332718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.334347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.335966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.336831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.336831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.340485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.342086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.344504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.345481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.345766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.346041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.346122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.346308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.349391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.349685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.349972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.350257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.355920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.356201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.356591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.360272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.360579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.360972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.364461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80395) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.364758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.365035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.365308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.365574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.365838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.366123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.366384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.366653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.366914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.367185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.367439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.367704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.367959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80395) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.368937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80396) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 23 13:08:18.369219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.369498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.369776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.371934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.372207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.372460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80396) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 23 13:08:18.407911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5602ab4329c0 pid: 80395 (default app) May 23 13:08:18.411438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05de7-7667-4f96-938a-e1b7d75fbd6c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:18.411729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05de7-7667-4f96-938a-e1b7d75fbd6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:18.412006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d05de7-7667-4f96-938a-e1b7d75fbd6c None None] PATH_INFO: `/v3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:18.412246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WSGI app 0 (mountpoint='') ready in 7 seconds on interpreter 0x5602ab4329c0 pid: 80396 (default app) May 23 13:08:18.416246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1/1] 10.222.0.70 () {50 vars in 791 bytes} [Mon May 23 13:08:18 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:27.933510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:27.934022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:27.934103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:27.938480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1/2] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:27 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:08:27.943842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:27.944217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:27.944399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:28.012617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: INFO dbcounter [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] Registered counter for database keystone May 23 13:08:28.034586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG oslo_db.sqlalchemy.engines [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80395) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 13:08:28.040794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writer thread running {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 13:08:28.219497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80395) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 13:08:28.219803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG passlib.handlers.bcrypt [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80395) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 13:08:28.276301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:28.277958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:28.405806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 2/3] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:27 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 463 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 13:08:28.592156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:28.661277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: INFO dbcounter [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] Registered counter for database keystone May 23 13:08:28.680496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG oslo_db.sqlalchemy.engines [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80396) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 23 13:08:28.687001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writer thread running {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 23 13:08:28.807993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efa3475-784d-45f1-83b6-a9579fdea093 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:28.812294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:28.812993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:28.813282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:28.813560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:28.821904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:28.874242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 13:08:28.874587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ddfd77c-016f-4076-bdb8-caf5b8201052 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:28.884933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 2/4] 10.222.0.70 () {58 vars in 1324 bytes} [Mon May 23 13:08:28 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:08:28.891012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:28.906098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3892ccc8-9507-47c4-81cb-43345cfb1c19 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:28.910851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:28.911976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:28.912266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:28.912541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] PATH_INFO: `/v3/projects/8db400662d744ef19141527de1d2418c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:28.914061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] RBAC: Authorizing `identity:get_project(project_id=8db400662d744ef19141527de1d2418c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:29.074292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING keystone.common.rbac_enforcer.enforcer [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 23 13:08:29.074915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b7fd5d3-f69c-4f56-88a7-64c037d09254 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:29.079310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 3/5] 10.222.0.70 () {56 vars in 1207 bytes} [Mon May 23 13:08:28 2022] GET /identity/v3/projects/8db400662d744ef19141527de1d2418c => generated 344 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:31.285516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97278b-f68f-417b-ba73-af2fbbc94336 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:31.286056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97278b-f68f-417b-ba73-af2fbbc94336 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:31.286056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e97278b-f68f-417b-ba73-af2fbbc94336 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:31.290221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 3/6] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:08:31.296290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:31.296722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:31.296886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:31.315616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:31.317190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:31.325246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 4/7] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:31 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 23 13:08:31.347396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97278b-f68f-417b-ba73-af2fbbc94336 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:31.360034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e97278b-f68f-417b-ba73-af2fbbc94336 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:31.364214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0588176f-4f69-4c40-bd43-994be8b95a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:31.364905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588176f-4f69-4c40-bd43-994be8b95a1b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:31.365189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588176f-4f69-4c40-bd43-994be8b95a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:31.365466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0588176f-4f69-4c40-bd43-994be8b95a1b admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:31.374222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588176f-4f69-4c40-bd43-994be8b95a1b admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:31.376227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0588176f-4f69-4c40-bd43-994be8b95a1b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:31.386107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 4/8] 10.222.0.70 () {58 vars in 1324 bytes} [Mon May 23 13:08:31 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:08:31.391881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:31.401629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b48329a-998e-459a-8bd9-8e034a1b506b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:31.406493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57e6773-2ad1-4ee9-8a70-8843ad147ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:31.407359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6773-2ad1-4ee9-8a70-8843ad147ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:31.407645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6773-2ad1-4ee9-8a70-8843ad147ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:31.408175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57e6773-2ad1-4ee9-8a70-8843ad147ef7 admin admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:31.409611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e6773-2ad1-4ee9-8a70-8843ad147ef7 admin admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:31.411440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57e6773-2ad1-4ee9-8a70-8843ad147ef7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:31.413046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 5/9] 10.222.0.70 () {56 vars in 1198 bytes} [Mon May 23 13:08:31 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 22 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:33.684526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea9e26-1727-4eaf-9955-1360bdb6873e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:33.685075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea9e26-1727-4eaf-9955-1360bdb6873e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:33.685183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ea9e26-1727-4eaf-9955-1360bdb6873e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:33.686478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 5/10] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:33.691741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:33.692143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:33.692314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:33.709674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:33.711771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:33.730881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab7593-019d-4d6c-86da-fdf5e56d7d7b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:33.731172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab7593-019d-4d6c-86da-fdf5e56d7d7b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:33.731454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aab7593-019d-4d6c-86da-fdf5e56d7d7b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:33.732885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 6/11] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:08:33.737930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:33.738221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:33.738502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:33.779299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG passlib.handlers.bcrypt [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80396) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 23 13:08:33.779725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG passlib.handlers.bcrypt [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80396) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 23 13:08:33.794578 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 6/12] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:33.818123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:33.826796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:33.828100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:33.842785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:33.928382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 7/13] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:33.929225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63da741f-c576-43c7-8f7c-7ff48e123a6b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:33.951276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:33.972385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:33.973237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:33.973521 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:33.973804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:33.987510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:34.008610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:34.010041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:34.010796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:08:34.011737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 7/14] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:08:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:08:34.018150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:34.061442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1619473-4689-46cd-8760-89a530b1da93 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.063745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:34.100056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.100993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.101289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.101574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.103425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:34.122366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4671a62-ff3f-40db-95ba-590863c6c9fc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.156332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.157143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.157447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.157983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.158744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:34.159053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.160535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.169854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.171375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.172938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 8/15] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:34.178495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:34.274392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 8/16] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:08:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:08:34.280087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:34.283160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:34.286441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b120d01-01c9-4a91-935c-3d6e4cc9824a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.329520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.330411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.330701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.330981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.332220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.334623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.337111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 9/17] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:34 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:34.342621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:34.363724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:34.383557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee31b7-ed47-41be-a1bf-4d28cc8c9380 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.418064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.418880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.419167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.419442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.438667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.440167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.457458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695444ce-5c92-4870-8e2b-bc87191f6706 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:08:34.461786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:34.463232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:34.464108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:08:34.464108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:08:34.464108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:08:34.464108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:34.464108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 9/18] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:08:34.469088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:34.496999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.497824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.498109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.498384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.499590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.501100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.502715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 10/19] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:34.508345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:34.535911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:34.584588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538740f0-9cfd-4611-9e94-f8aa9045b778 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.601197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:34.620805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.621842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.621842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.622040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.622766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:34.623074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.624488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe39e258-b50b-4bd8-93c6-8a487c406c48 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.624960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41332e42-06a4-43c0-8165-cc93e46eb192 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:34.645701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 10/20] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:34.665462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:34.667007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:34.667302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:34.667579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:34.668468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.domain.id=default, user.id=5acd460bee394e719cdafd0e39f18395) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:34.668823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:34.670356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2d8c64-9b7d-4b3c-811f-0ab59d2f340c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:34.678397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 11/21] 10.222.0.70 () {56 vars in 1322 bytes} [Mon May 23 13:08:34 2022] GET /identity/v3/role_assignments?user.id=5acd460bee394e719cdafd0e39f18395&scope.domain.id=default&role.id=33de09388f974cd391e13bb4f2502a52 => generated 233 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:37.247907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c272e6-3c7f-4f37-b0f9-98261db4ade1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.248411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c272e6-3c7f-4f37-b0f9-98261db4ade1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.248515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c272e6-3c7f-4f37-b0f9-98261db4ade1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.250130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 11/22] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:37.258687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84018cf-c9f5-46a2-9e0d-193e05543f3d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.259255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84018cf-c9f5-46a2-9e0d-193e05543f3d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.259467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84018cf-c9f5-46a2-9e0d-193e05543f3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.287325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f84018cf-c9f5-46a2-9e0d-193e05543f3d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:37.288608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f84018cf-c9f5-46a2-9e0d-193e05543f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:37.383316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 12/23] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:37.422366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c272e6-3c7f-4f37-b0f9-98261db4ade1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:37.569986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c4d3-5b72-453d-80a4-4db512466a37 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.570290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c4d3-5b72-453d-80a4-4db512466a37 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.570468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b398c4d3-5b72-453d-80a4-4db512466a37 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.572404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 13/24] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:08:37.579925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582ca2b-448f-4fcd-ba70-f1a787fa5f97 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.580210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582ca2b-448f-4fcd-ba70-f1a787fa5f97 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.580487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0582ca2b-448f-4fcd-ba70-f1a787fa5f97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.612863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0582ca2b-448f-4fcd-ba70-f1a787fa5f97 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:37.614123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0582ca2b-448f-4fcd-ba70-f1a787fa5f97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:37.753486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c272e6-3c7f-4f37-b0f9-98261db4ade1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:37.757996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 14/25] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:37.763864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4aece-0a90-48e9-a2b3-ac474d429187 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.764154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4aece-0a90-48e9-a2b3-ac474d429187 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.764436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a4aece-0a90-48e9-a2b3-ac474d429187 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.766224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 15/26] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:37 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:37.773966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.774255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.774538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.792957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:37.793701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:37.806670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:37.807898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:37.813371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:37.814488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:37.815101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:37.815570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:37.849255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:37.858456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:37.859322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:37.860069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 12/27] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:08:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 438 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:37.860726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:37.889774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 16/28] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:37.890465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:37.989380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2e330a-9216-4cad-bb4c-6693854760f8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:37.998536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a13174d9-ba4b-4b84-8bb6-a21da1005891 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:38.027349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6309ab-c71b-43a1-a19d-4ee4f2418b40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.028038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6309ab-c71b-43a1-a19d-4ee4f2418b40 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.028321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6309ab-c71b-43a1-a19d-4ee4f2418b40 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.028613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6309ab-c71b-43a1-a19d-4ee4f2418b40 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.029383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6309ab-c71b-43a1-a19d-4ee4f2418b40 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.030792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6309ab-c71b-43a1-a19d-4ee4f2418b40 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.035432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.036202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.036495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.036806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.037626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:38.038006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.039823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6684390a-635e-4849-a3ff-38e58e94a5b9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.041558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:38.052649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 17/29] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:38.052649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021c331-5af4-455f-8234-9c5d0c31e9b3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.052649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021c331-5af4-455f-8234-9c5d0c31e9b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.052649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b021c331-5af4-455f-8234-9c5d0c31e9b3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.053430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 18/30] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:38.053590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021c331-5af4-455f-8234-9c5d0c31e9b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:38.165514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b021c331-5af4-455f-8234-9c5d0c31e9b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:38.222040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.222900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.223187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.223465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.236364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.238005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:08:38.267478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:38.269070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:38.270271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:08:38.270271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:08:38.270271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:08:38.270271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:08:38.270271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 19/31] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:38.270271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:38.311464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 13/32] 10.222.0.70 () {60 vars in 1131 bytes} [Mon May 23 13:08:37 2022] POST /identity/v3/roles => generated 219 bytes in 447 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:38.311464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.311464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.311464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.351500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:38.365736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:38.367336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:38.439321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:38.463702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e115e3e2-1fe4-4fb4-8b6b-fe09abf8a5c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:38.487023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 14/33] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:38.488042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:38.565931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.567216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.567717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.568209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.577777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.581434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.583569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 20/34] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:38 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:38.590559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:38.633898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fd472f-5807-437c-a84a-d5e072f9749f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:38.684791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.685491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.685780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.686062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.686816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:38.687128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.688695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.702175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 15/35] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:38.702767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:38.724591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5de44106-d447-41d6-ab2c-47005b0bc9ed None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:38.732647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:38.745727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:38.765476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.766518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.767017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.767543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.768527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.770229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.829950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f72cd1-848d-401d-9795-a0d6196ab238 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:38.867444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:38.869289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:38.869788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:38.870238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:38.871999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:38.873672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:38.877882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 16/36] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:38.878971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:38.996917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:38.998879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 21/37] 10.222.0.70 () {60 vars in 1131 bytes} [Mon May 23 13:08:38 2022] POST /identity/v3/roles => generated 225 bytes in 409 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:39.002193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:39.004274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dcef84-3909-43ab-acd7-6fa15e14295f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:39.056082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515b17ed-c642-44a7-8da7-6c4c2ba38d48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:39.057577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b17ed-c642-44a7-8da7-6c4c2ba38d48 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.058095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b17ed-c642-44a7-8da7-6c4c2ba38d48 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.058652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515b17ed-c642-44a7-8da7-6c4c2ba38d48 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.059643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b17ed-c642-44a7-8da7-6c4c2ba38d48 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:39.061537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515b17ed-c642-44a7-8da7-6c4c2ba38d48 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:39.131822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72bf1b6f-a1ce-45f5-a8c3-09dc909743aa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:39.169599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76458cce-9665-45bd-bf51-17b27c626ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:39.170511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458cce-9665-45bd-bf51-17b27c626ea2 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.170802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458cce-9665-45bd-bf51-17b27c626ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.171086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76458cce-9665-45bd-bf51-17b27c626ea2 None admin] PATH_INFO: `/v3/domains/default/users/5acd460bee394e719cdafd0e39f18395/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.173465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76458cce-9665-45bd-bf51-17b27c626ea2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=5acd460bee394e719cdafd0e39f18395, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:39.175089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76458cce-9665-45bd-bf51-17b27c626ea2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:39.210724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 17/38] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:08:38 2022] POST /identity/v3/projects => generated 301 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:39.212350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28292f3-31c7-4347-99ba-26072d6b37ee None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.212671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28292f3-31c7-4347-99ba-26072d6b37ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.212954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28292f3-31c7-4347-99ba-26072d6b37ee None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.214447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 18/39] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:39.215689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f58c6-c4ac-405a-8c42-dba10ee92ddd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.215981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f58c6-c4ac-405a-8c42-dba10ee92ddd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.216266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f58c6-c4ac-405a-8c42-dba10ee92ddd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.220667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 19/40] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:39.222856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.223153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.223434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.243342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:39.245371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:39.284727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 22/41] 10.222.0.70 () {58 vars in 1361 bytes} [Mon May 23 13:08:38 2022] PUT /identity/v3/domains/default/users/5acd460bee394e719cdafd0e39f18395/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:08:39.284727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.284727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.284727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.313695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:39.316283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:39.336100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 20/42] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:39.358251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:39.360551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:39.418066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 23/43] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:39.448287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:39.463569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:39.504935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3cbfe9-4f7d-46e4-a659-9ec381f83c00 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:39.557278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f265e5b5-50bf-40fb-a6b4-4c2f865a278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:39.557890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e5b5-50bf-40fb-a6b4-4c2f865a278e None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.558182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e5b5-50bf-40fb-a6b4-4c2f865a278e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.558463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f265e5b5-50bf-40fb-a6b4-4c2f865a278e None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.559224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265e5b5-50bf-40fb-a6b4-4c2f865a278e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:39.560694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f265e5b5-50bf-40fb-a6b4-4c2f865a278e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:39.586916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164c315b-88cb-40c7-8584-97a7ea428fcd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:39.622535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:39.623343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.623636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.623914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.625220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:39.626726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:39.628356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 24/44] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:39.633838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:39.661976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 21/45] 10.222.0.70 () {60 vars in 1131 bytes} [Mon May 23 13:08:39 2022] POST /identity/v3/roles => generated 223 bytes in 302 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:39.671317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:39.751960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170119d1-6733-4f0d-a53a-00a100bc786a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:39.793153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374349a2-9ff7-4a33-b1d0-6a9b5bfcd439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:39.793851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374349a2-9ff7-4a33-b1d0-6a9b5bfcd439 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:39.794144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374349a2-9ff7-4a33-b1d0-6a9b5bfcd439 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:39.794426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374349a2-9ff7-4a33-b1d0-6a9b5bfcd439 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:39.795181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374349a2-9ff7-4a33-b1d0-6a9b5bfcd439 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:39.796742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374349a2-9ff7-4a33-b1d0-6a9b5bfcd439 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:39.936241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 25/46] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:08:39 2022] POST /identity/v3/projects => generated 312 bytes in 303 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:08:41.572620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b846-3980-475c-a967-8d05caf9275c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:41.573151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b846-3980-475c-a967-8d05caf9275c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:41.573151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b846-3980-475c-a967-8d05caf9275c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:41.574469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 22/47] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:41.579959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:41.580414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:41.580536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:41.598185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:41.599158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:41.668158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 26/48] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:41.692519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b846-3980-475c-a967-8d05caf9275c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:41.796428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b846-3980-475c-a967-8d05caf9275c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:41.833342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:41.834068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:41.834354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:41.834631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:41.838073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:41.846325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:41.854828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:41.855835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:41.856642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 23/49] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:08:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:41.861966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:41.879137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:41.961018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2153c305-1fa0-4a01-9713-953e88957524 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:41.998566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:42.003601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.003601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.003601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.003601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:42.003601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:42.003601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:42.012904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 27/50] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:42.018850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:42.123327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81debec-b6ff-4f23-bed3-03d8477fea64 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:42.158840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040e982c-ee28-4404-953d-e376c2702cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:42.159726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e982c-ee28-4404-953d-e376c2702cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.160062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e982c-ee28-4404-953d-e376c2702cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.160289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040e982c-ee28-4404-953d-e376c2702cb1 None admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.161766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e982c-ee28-4404-953d-e376c2702cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:42.163230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040e982c-ee28-4404-953d-e376c2702cb1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:42.164901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 24/51] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:42 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:42.170341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:42.196100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6b56b-cd78-4dcd-876b-6cd9233b2cc0 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.197276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6b56b-cd78-4dcd-876b-6cd9233b2cc0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.197563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a6b56b-cd78-4dcd-876b-6cd9233b2cc0 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.199176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 25/52] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:08:42.204125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.204414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.204729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.222060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:42.223278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:42.262618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:42.284901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:42.285628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a368240-3aa8-4005-b32f-219b62bb430c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:42.293511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 26/53] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:42.316843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:42.323201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:42.324022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.324309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.324612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.325848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:42.327343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:42.329059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 28/54] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:42.334791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:42.419682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba592598-e048-4729-8d3e-e3478e15f455 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:42.438167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f25c2544-712c-454d-ab54-54655624ed22 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:42.455874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:42.456719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.457012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.457295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.458481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:42.459960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:42.461632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 27/55] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:42.467162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:42.472322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:42.473022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.473314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.473598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.474447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.domain.id=default, user.id=5acd460bee394e719cdafd0e39f18395) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:42.474787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:42.476259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46ad4e2-7a83-48fd-a0c9-9b4c2d2b4720 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:42.483115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 29/56] 10.222.0.70 () {56 vars in 1322 bytes} [Mon May 23 13:08:42 2022] GET /identity/v3/role_assignments?user.id=5acd460bee394e719cdafd0e39f18395&scope.domain.id=default&role.id=33de09388f974cd391e13bb4f2502a52 => generated 530 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:42.534728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:42.576945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d0bb5c-bf33-4b6c-a8b1-e28af659a954 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:42.614337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15ff1f7-af70-47c3-bc01-7bca878434cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:42.615020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ff1f7-af70-47c3-bc01-7bca878434cd None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:42.615305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ff1f7-af70-47c3-bc01-7bca878434cd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:42.615587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15ff1f7-af70-47c3-bc01-7bca878434cd None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:42.616313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ff1f7-af70-47c3-bc01-7bca878434cd None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:42.621952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15ff1f7-af70-47c3-bc01-7bca878434cd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:42.736778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 28/57] 10.222.0.70 () {60 vars in 1140 bytes} [Mon May 23 13:08:42 2022] POST /identity/v3/projects => generated 298 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:08:44.976007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ebc1-863f-4005-9507-b53031a5df61 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:44.976529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ebc1-863f-4005-9507-b53031a5df61 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:44.976676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0572ebc1-863f-4005-9507-b53031a5df61 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:44.978278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 30/58] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:44.983699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:44.984078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:44.984262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:45.001193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:45.002449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:45.072140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 29/59] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:45.096970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ebc1-863f-4005-9507-b53031a5df61 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:45.110583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:45.201177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0572ebc1-863f-4005-9507-b53031a5df61 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:45.234686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf339527-f6d7-45b8-b08f-8cd17cb9f1ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:45.235571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf339527-f6d7-45b8-b08f-8cd17cb9f1ef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:45.235866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf339527-f6d7-45b8-b08f-8cd17cb9f1ef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:45.236149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf339527-f6d7-45b8-b08f-8cd17cb9f1ef None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:45.237365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf339527-f6d7-45b8-b08f-8cd17cb9f1ef None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:45.238828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf339527-f6d7-45b8-b08f-8cd17cb9f1ef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:45.240494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 31/60] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:45.247135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:45.292864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:45.346966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c364ee-1855-4b09-a009-0adeae8f614e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:45.380711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b2ef0-ae3f-46d5-8832-07421a7d17e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:45.381380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b2ef0-ae3f-46d5-8832-07421a7d17e1 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:45.381673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b2ef0-ae3f-46d5-8832-07421a7d17e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:45.381955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b2ef0-ae3f-46d5-8832-07421a7d17e1 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:45.382666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b2ef0-ae3f-46d5-8832-07421a7d17e1 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:45.385301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b2ef0-ae3f-46d5-8832-07421a7d17e1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:45.475823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 30/61] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:08:45 2022] POST /identity/v3/users => generated 275 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:48.912918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fcb8f-192b-405b-9fbe-0904f8b4a0e0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:48.913445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fcb8f-192b-405b-9fbe-0904f8b4a0e0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:48.913521 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5fcb8f-192b-405b-9fbe-0904f8b4a0e0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:48.914960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 32/62] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:48.920347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:48.921285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:48.921773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:48.943424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:48.944992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:48.950452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded46bae-ce0a-49f3-9f0d-875470a1c75f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:48.951005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded46bae-ce0a-49f3-9f0d-875470a1c75f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:48.951505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded46bae-ce0a-49f3-9f0d-875470a1c75f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:48.953119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 33/63] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:08:48.958561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:48.959127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:48.959629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:48.982665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:48.983991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:49.034861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 31/64] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:49.060849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.097445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.140428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 34/65] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:49.165857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.176955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.228527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee339788-aaf7-4cbe-a322-2e1704cf78b1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.304959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5349f-289f-4c12-a75b-7dd23bb82495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.306364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5349f-289f-4c12-a75b-7dd23bb82495 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.306653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5349f-289f-4c12-a75b-7dd23bb82495 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.306928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5349f-289f-4c12-a75b-7dd23bb82495 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.328494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5349f-289f-4c12-a75b-7dd23bb82495 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-69b5349f-289f-4c12-a75b-7dd23bb82495 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:49.337180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:49.338211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:49.339087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:49.341084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 32/66] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:08:49.347367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eea48-b73d-496c-b485-85995ca7d881 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.347832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eea48-b73d-496c-b485-85995ca7d881 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.348247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8eea48-b73d-496c-b485-85995ca7d881 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.350261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 33/67] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:49.351137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eea48-b73d-496c-b485-85995ca7d881 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.355541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97fd9fc2-c31f-4add-ac35-b77bf6aba124 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.404085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88294e1-6ec3-450b-b31e-72089c877886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.405202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88294e1-6ec3-450b-b31e-72089c877886 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.406233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88294e1-6ec3-450b-b31e-72089c877886 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.406922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88294e1-6ec3-450b-b31e-72089c877886 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.421427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88294e1-6ec3-450b-b31e-72089c877886 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c88294e1-6ec3-450b-b31e-72089c877886 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:49.432746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:49.433811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:49.434755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:08:49.438324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 35/68] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:08:49.439633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4999bf-275a-452c-80a4-53c3d1a8cd0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.439921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4999bf-275a-452c-80a4-53c3d1a8cd0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.440200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4999bf-275a-452c-80a4-53c3d1a8cd0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.458469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ea4999bf-275a-452c-80a4-53c3d1a8cd0f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:49.459674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea4999bf-275a-452c-80a4-53c3d1a8cd0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:49.468317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.480780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8eea48-b73d-496c-b485-85995ca7d881 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.518757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.519438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.519727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.520008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.520797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:49.521111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:49.522520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:49.530797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.535015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 34/69] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:49.544116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.544243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 36/70] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:49.544243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dce96-1596-4fe5-ab61-eaa4b989c436 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.544243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dce96-1596-4fe5-ab61-eaa4b989c436 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.544243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dce96-1596-4fe5-ab61-eaa4b989c436 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.545182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 37/71] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:49.545907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dce96-1596-4fe5-ab61-eaa4b989c436 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.647660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789616e5-6885-43a0-a644-ae7e91a8e9cf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.650653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dce96-1596-4fe5-ab61-eaa4b989c436 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.683312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.683918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.684209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.684488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.685278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:49.685593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:49.686103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481220eb-1f03-4406-9128-c3ca72c5f3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.686978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:49.687066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481220eb-1f03-4406-9128-c3ca72c5f3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.687269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481220eb-1f03-4406-9128-c3ca72c5f3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.687560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481220eb-1f03-4406-9128-c3ca72c5f3a4 None admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.688802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481220eb-1f03-4406-9128-c3ca72c5f3a4 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:49.690267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481220eb-1f03-4406-9128-c3ca72c5f3a4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:49.691922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 38/72] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:49.693096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a1eb-6bc6-4dd7-b49a-b314d6bf9d22 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.693390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a1eb-6bc6-4dd7-b49a-b314d6bf9d22 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.693674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b9a1eb-6bc6-4dd7-b49a-b314d6bf9d22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.700085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 35/73] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:49.700977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.711958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-61b9a1eb-6bc6-4dd7-b49a-b314d6bf9d22 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:49.712885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-61b9a1eb-6bc6-4dd7-b49a-b314d6bf9d22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:49.717401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.774615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.782453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 39/74] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:49.783777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395047a5-3ab3-4af0-b063-2d67c3664cb2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.784075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395047a5-3ab3-4af0-b063-2d67c3664cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.784395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395047a5-3ab3-4af0-b063-2d67c3664cb2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.788727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 40/75] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:49.788727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395047a5-3ab3-4af0-b063-2d67c3664cb2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.808707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c92017a-781a-47a7-9380-37387ef65e09 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.844215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.845130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.845379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.845678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.856045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:49.864136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:49.865042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:49.866016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 36/76] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:49.868983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:49.893567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395047a5-3ab3-4af0-b063-2d67c3664cb2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:49.939404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:49.940063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:49.940269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:49.940606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:49.962407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:49.986831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949ac666-30b0-4e00-b552-4a927e52a46c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.013295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.019986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.021526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.028842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd89b05e-d628-471c-a168-98c39bd7dfab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.028842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b05e-d628-471c-a168-98c39bd7dfab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.028842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b05e-d628-471c-a168-98c39bd7dfab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.028842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd89b05e-d628-471c-a168-98c39bd7dfab None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.039277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b05e-d628-471c-a168-98c39bd7dfab None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.041224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd89b05e-d628-471c-a168-98c39bd7dfab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.042926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 37/77] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:50.044133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.044426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.044754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.061421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:50.062628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:50.124109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 41/78] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:08:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:08:50.124896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.132618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 38/79] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:50.134550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.207806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.224909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.231957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38568cc7-e62e-4dd9-a544-5f4e71f204a1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.248243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b437069-ea45-4be0-b679-84c35d60cc1f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.266381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.267767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.267767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.267767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.268921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.270408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.272035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 42/80] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:50.272830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.293971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.294665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.295033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.295321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.296089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:50.296405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.299657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.311926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 39/81] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:50.312821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.388855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b395af8-2928-47f7-b5e0-9ebd13a5e68a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.420647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1f6dde-6907-4952-8555-a679637cec0b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.423858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.424514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.424832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.425117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.447421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.455652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.462194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.462194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.462194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.462194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.462637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.462637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.464343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 40/82] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:50.465156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.494803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.496305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.565417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc36ffe7-e6d5-4dd3-89a7-7f37d4277381 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.594128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 43/83] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:08:50.594969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.600928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.601899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.602391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.602952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.629367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:50.640170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:50.641128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:50.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:50.643050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 41/84] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:50.643869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.667849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.706144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd855e-3e8d-4e03-88b1-85f92f9fae5a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.706333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.739937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.740541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.740882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.741165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.741876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.743533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.754902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5a845d-90be-406e-94b3-43197edc6c26 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.792161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.793485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.793791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.794079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.797902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.800677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:50.803808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 42/85] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:50.805779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.838148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 44/86] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:08:50 2022] POST /identity/v3/users => generated 289 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:50.838783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:50.937793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc80c984-cafb-42b1-9ac5-3fd2c36f3426 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.962094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2060c9a7-7ba0-401e-b8eb-0ebf1403e700 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:50.969796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:50.984425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:50.987573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:50.987882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:50.988168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:50.989114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=5acd460bee394e719cdafd0e39f18395) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:50.989588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:50.992471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.001639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 43/87] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/role_assignments?user.id=5acd460bee394e719cdafd0e39f18395&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:51.002364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.047148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.075386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.076477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.076887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.077169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.078697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.080255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.083103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 45/88] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:50 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:51.083467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.171246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c84ffc-00ff-439c-9729-ae2a55a714e1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:51.238594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.239300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.239594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.239866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.240666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:51.241075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.243922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.249936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ba4171-b1be-40bc-96d4-6cc098e3a667 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:51.254561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 44/89] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:51.255603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.293272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.294442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.295298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.296021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.317044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.395008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7017379-e055-4d3b-80da-337b0f6806a6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:51.412478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.432366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.434524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.436065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.436777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.437108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.437391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.438341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=d43e4a70df3c4919a2e14f9ad3265e10) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:51.438657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.440112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.446535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 45/90] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/role_assignments?user.id=d43e4a70df3c4919a2e14f9ad3265e10&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:51.447432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.572453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9449434e-8199-4694-803c-2e911d9ffe59 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:51.582973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 46/91] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:08:51.586593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.590203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.613781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.614730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.615052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.615429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.617535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.620141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.623361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 46/92] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:51.629819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.706175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.723007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125f5d3d-4077-42c8-89b0-5bcebca692a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:51.805452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.806439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.806733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.807011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.808310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.809835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:51.811791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 47/93] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:51.819962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4df22f8-475f-4a9f-82e1-5dfc0fb2fdb4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:51.821945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:51.860287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:51.861002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:51.861288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:51.862060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:51.917883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:51.943776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:51.945281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:52.063715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 47/94] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:08:52.069994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:52.083978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf777b63-6662-42ce-ba63-a1d2db1d427c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:52.169379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:52.169995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:52.170284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:52.170562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:52.171411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] RBAC: Adding query filter params (role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=d43e4a70df3c4919a2e14f9ad3265e10) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:52.171725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:52.173227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:52.187503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 48/95] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:08:51 2022] GET /identity/v3/role_assignments?user.id=d43e4a70df3c4919a2e14f9ad3265e10&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=a7511c9c66e546dda3a737b069282dae => generated 259 bytes in 366 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:52.201260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:52.250509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ee77dd-4ae6-4fb5-899d-7ae3f527ab59 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:52.303555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b3f325-e8ed-47a9-99cd-f2696cce9e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:52.304470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f325-e8ed-47a9-99cd-f2696cce9e0d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:52.305360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f325-e8ed-47a9-99cd-f2696cce9e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:52.305678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b3f325-e8ed-47a9-99cd-f2696cce9e0d None admin] PATH_INFO: `/v3/projects/27a6d47620824971878ce596fab02fa4` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:52.309588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3f325-e8ed-47a9-99cd-f2696cce9e0d None admin] RBAC: Authorizing `identity:get_project(project_id=27a6d47620824971878ce596fab02fa4)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:52.311175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b3f325-e8ed-47a9-99cd-f2696cce9e0d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:52.313154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 48/96] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:52 2022] GET /identity/v3/projects/27a6d47620824971878ce596fab02fa4 => generated 312 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:52.319422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:52.355681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:52.469093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf63546-028b-47a4-9ba5-ebe97919434d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:52.507540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:52.508282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:52.508588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:52.508888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:52.509751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=27a6d47620824971878ce596fab02fa4, user.id=d43e4a70df3c4919a2e14f9ad3265e10) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:52.510075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:52.511573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a355e09e-ce66-4df9-a194-92e0c0a40832 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:52.519915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 49/97] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:08:52 2022] GET /identity/v3/role_assignments?user.id=d43e4a70df3c4919a2e14f9ad3265e10&scope.project.id=27a6d47620824971878ce596fab02fa4&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 259 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:54.228944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37776c99-9997-4402-bff9-138c32fbfdad None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.229906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37776c99-9997-4402-bff9-138c32fbfdad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.230387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37776c99-9997-4402-bff9-138c32fbfdad None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.231999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 49/98] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:54.237688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.238573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.239113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.265599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:54.267164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:54.358081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 50/99] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:54.429230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed638a-ea1c-4d93-b494-b0f5bd5cba08 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.429892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed638a-ea1c-4d93-b494-b0f5bd5cba08 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.430397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ed638a-ea1c-4d93-b494-b0f5bd5cba08 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.432290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 50/100] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:54.434362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:54.451612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:54.453353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.453649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.453925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.486106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:54.487327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:54.643427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 51/101] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:54.666892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3393f841-615a-42fa-9337-cddeaad2d15b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:54.713077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:54.737270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:54.738104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.738390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.738665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.769741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:54.778120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:54.779061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:08:54.779814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 51/102] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:54.787370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:54.847674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:54.879784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb213e3-5c87-45ec-a0f8-2b1f4a4e2fe1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:54.924710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:54.925852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:54.926678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.927008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.927238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.941100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:54.949578 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3a001a-0fe6-4b5a-aa2e-5ac4fc6d87ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:54.957070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:54.958047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:54.958787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 52/103] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:08:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:54.978549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:54.993107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:54.993801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:54.994090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:54.994373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:54.995141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:54.995454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:54.997034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.006098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 52/104] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:08:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:55.011673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.137103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21195879-4042-418d-9a11-abac59f91c3e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.143450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6cff5e-ef35-427a-83dd-eb2493d96ab9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.174052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.175074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.175690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.176177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.177229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:55.177745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.179363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e2b4d5-baa3-488e-90c8-591d89cfebd7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.182476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.184018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a15fc65-ecdd-48dc-ba3e-800a21ace9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.185866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15fc65-ecdd-48dc-ba3e-800a21ace9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.186423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15fc65-ecdd-48dc-ba3e-800a21ace9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.186911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a15fc65-ecdd-48dc-ba3e-800a21ace9d8 None admin] PATH_INFO: `/v3/users/19e4d96060764900a614e99e723b0bbf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.190861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.197728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 53/105] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:55.197728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19511caa-206a-4c9a-a06e-928d340fbe47 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.197728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19511caa-206a-4c9a-a06e-928d340fbe47 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.197728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19511caa-206a-4c9a-a06e-928d340fbe47 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.197728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 54/106] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:55.198800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ece583-5123-4a2e-92d5-b25cae319084 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.199091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ece583-5123-4a2e-92d5-b25cae319084 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.199369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ece583-5123-4a2e-92d5-b25cae319084 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.201514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 55/107] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:55.203317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.203317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.203569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.210515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15fc65-ecdd-48dc-ba3e-800a21ace9d8 None admin] RBAC: Authorizing `identity:get_user(user_id=19e4d96060764900a614e99e723b0bbf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.212294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a15fc65-ecdd-48dc-ba3e-800a21ace9d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.214299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 53/108] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/users/19e4d96060764900a614e99e723b0bbf => generated 289 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:55.215881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.216377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.216893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.220996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:55.222194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:55.236617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:55.238388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:55.298662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 56/109] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:55.299801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.323358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 54/110] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:55.324743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.422563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0690a18-92cf-40ca-b3df-debf99e39b8b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.433008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba8f8fd-2f17-4404-827e-b27e513671a4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.452881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.460631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.462361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.467022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.467022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.467022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.467022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.467022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.468453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 57/111] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:55.469419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.479354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.480250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.480754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.481180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.559373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.561303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.589643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7033b0e5-9eec-430c-b7cc-b62911ab2b14 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.630475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.631862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.632386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.633448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.645570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:55.655100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:55.656016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:55.657070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:55.659780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 58/112] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:55.661015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.674511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 55/113] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:08:55.675486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.703311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.729946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.778644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c454a3af-d946-4db2-beaf-c27c8e616434 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.787880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd428b38-6b0d-4cde-9b67-1463f2cc411b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.818232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.819090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.819537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.819818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.821976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.823411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.824826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.825085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 59/114] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:55.825877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.826169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.826460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.826745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.827979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.830746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.837979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 56/115] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:55.837979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.937510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.938420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4f2800-80fc-4e38-a507-01a3171bcf32 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.947311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557516e6-a649-461a-80eb-a8e7730dc4e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:55.965018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:55.975774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.977028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.977325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.977897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.978472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:55.978790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.980239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d188d99-c4a6-4417-9fe4-ec1edc6892ec None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.986596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:55.987547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.987789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.988070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.994514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:55.994693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 60/116] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:55.994693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191f5c9-e72b-4d49-b327-506194db774a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:55.994823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:55.994886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191f5c9-e72b-4d49-b327-506194db774a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:55.994886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3191f5c9-e72b-4d49-b327-506194db774a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:55.994886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 61/117] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:55.995133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 57/118] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:55 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:55.995133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:55.997308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191f5c9-e72b-4d49-b327-506194db774a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.109215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf668ebd-5284-41da-80e5-fdd27f5a7170 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:56.114191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3191f5c9-e72b-4d49-b327-506194db774a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:56.145567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:56.146472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.146756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.147025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e/users/5acd460bee394e719cdafd0e39f18395/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.149679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] RBAC: Authorizing `identity:create_grant(project_id=66be157488cf4d29902de7af0526cf6e, user_id=5acd460bee394e719cdafd0e39f18395, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:56.151242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:56.152290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea851df-9c21-4e26-afd3-7677b6934f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:56.152994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea851df-9c21-4e26-afd3-7677b6934f8e None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.153295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea851df-9c21-4e26-afd3-7677b6934f8e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.153583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea851df-9c21-4e26-afd3-7677b6934f8e None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.154320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea851df-9c21-4e26-afd3-7677b6934f8e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:56.155730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea851df-9c21-4e26-afd3-7677b6934f8e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:56.219004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 58/119] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:08:55 2022] PUT /identity/v3/projects/66be157488cf4d29902de7af0526cf6e/users/5acd460bee394e719cdafd0e39f18395/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:08:56.219838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.250673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 62/120] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:08:55 2022] POST /identity/v3/projects => generated 302 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:08:56.250673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.250673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.250673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.259396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:56.266136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:56.267710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:56.307682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:56.356483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 63/121] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:56.357823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.400910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10b329-89ca-4783-9ac5-eacb48205929 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:56.465735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b8bdaa-3c51-43f0-a654-8f494ced61f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:56.466945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8bdaa-3c51-43f0-a654-8f494ced61f8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.467417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8bdaa-3c51-43f0-a654-8f494ced61f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.467867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b8bdaa-3c51-43f0-a654-8f494ced61f8 None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.469426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8bdaa-3c51-43f0-a654-8f494ced61f8 None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:56.471199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b8bdaa-3c51-43f0-a654-8f494ced61f8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:56.473119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 59/122] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:56 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:56.476008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c009208-c253-48ce-9194-2828ee29c263 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.476782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c009208-c253-48ce-9194-2828ee29c263 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.478468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c009208-c253-48ce-9194-2828ee29c263 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.478927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 60/123] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:56.479667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c009208-c253-48ce-9194-2828ee29c263 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.516107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca813ad3-beaa-4923-bf37-59fd5333d8fe None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:56.559032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:56.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.560506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.560988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.562016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] RBAC: Adding query filter params (role.id=30181561a7e54d82b14a5ea28bafe082, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=19e4d96060764900a614e99e723b0bbf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:56.562515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:56.564185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e978ae-eb91-4d5d-80ed-41e155d5af35 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:56.571182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 64/124] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:08:56 2022] GET /identity/v3/role_assignments?user.id=19e4d96060764900a614e99e723b0bbf&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=30181561a7e54d82b14a5ea28bafe082 => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:56.572842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.573325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.573787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.609160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:56.610483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:56.622704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:56.645909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:56.657759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c009208-c253-48ce-9194-2828ee29c263 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:56.699084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 65/125] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:56.700076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.745582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:56.746779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.747269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.747768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.780216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:56.799119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:56.800029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:08:56.800809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:08:56.803364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 61/126] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:08:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:56.806360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.818791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c675e7-dc50-4076-ace5-b5c0783aa917 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:56.863138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:56.864029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:56.864317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:56.864617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:56.868483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:56.868483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:56.871070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 66/127] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:56 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:56.871852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:56.923146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:57.003449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e624918-0272-46cc-b516-df77df4b375a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.043743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.044792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.045155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.045869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.047029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:57.047407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.048465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf54aa2-4816-4e0e-a36f-596c200ffdc4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.049175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.060958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 67/128] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:08:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:57.061946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:57.169568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:57.183986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.184834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.187961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604920f-0b0d-41b0-8abe-93e68c0f1f09 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.188073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.188073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.217652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:57.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:57.232295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:08:57.233085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:08:57.238592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 62/129] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 433 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:57.240977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:57.258894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.260155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.260693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.261172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e/users/d43e4a70df3c4919a2e14f9ad3265e10/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.266298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=66be157488cf4d29902de7af0526cf6e, user_id=d43e4a70df3c4919a2e14f9ad3265e10, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.268030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.268919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:57.368293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 68/130] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:08:57 2022] PUT /identity/v3/projects/66be157488cf4d29902de7af0526cf6e/users/d43e4a70df3c4919a2e14f9ad3265e10/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:08:57.369879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:57.393460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72802ab-5637-4f4d-a702-6694a1554e59 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.485559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.486411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.486718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.487061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.488980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.490017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.491695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 63/131] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:57 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:57.498016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:57.528281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08129a8d-bb9b-4f0e-bf9c-7d56943751f0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.575883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.576941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.577656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.579044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.580037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:57.580593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.582264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.596305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 69/132] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:08:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:57.604604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:57.634002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:57.636751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:57.692522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99455d7d-e8cc-4cdd-b552-2042c7befecf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.763893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.764726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.765014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.765293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.766632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.768108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.770330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 64/133] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:57 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:08:57.776213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:57.840507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a98637d-99d7-4591-bc45-9f06da76e7f7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.895968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411a61dc-45de-448c-bcfb-5dd01717736e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:57.931305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.932211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.932498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.932818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.935232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:57.936282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:57.936744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:57.937160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e/users/d43e4a70df3c4919a2e14f9ad3265e10/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:57.937283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.938776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.939781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] RBAC: Authorizing `identity:create_grant(project_id=66be157488cf4d29902de7af0526cf6e, user_id=d43e4a70df3c4919a2e14f9ad3265e10, role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:57.940774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 70/134] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:57 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:57.941675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:57.946084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:58.025727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 65/135] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:08:57 2022] PUT /identity/v3/projects/66be157488cf4d29902de7af0526cf6e/users/d43e4a70df3c4919a2e14f9ad3265e10/roles/a7511c9c66e546dda3a737b069282dae => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:08:58.045763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:58.066259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55800c9c-cb07-4f36-8a20-6a1d9e203dee None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:58.108276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76049c4b-bd6a-43f1-b425-d937dfcc0c5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:58.109469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049c4b-bd6a-43f1-b425-d937dfcc0c5c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:58.109896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049c4b-bd6a-43f1-b425-d937dfcc0c5c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:58.110325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76049c4b-bd6a-43f1-b425-d937dfcc0c5c None admin] PATH_INFO: `/v3/projects/27a6d47620824971878ce596fab02fa4` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:58.111763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76049c4b-bd6a-43f1-b425-d937dfcc0c5c None admin] RBAC: Authorizing `identity:get_project(project_id=27a6d47620824971878ce596fab02fa4)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:58.113459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76049c4b-bd6a-43f1-b425-d937dfcc0c5c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:58.115463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 71/136] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:08:57 2022] GET /identity/v3/projects/27a6d47620824971878ce596fab02fa4 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:58.122892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:58.127346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:58.260686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a71c18a-8be5-48d6-9c3c-532561b44996 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:58.307231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f3e103-c2e4-4368-9dd1-b1cc45790463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:58.308069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3e103-c2e4-4368-9dd1-b1cc45790463 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:58.308358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3e103-c2e4-4368-9dd1-b1cc45790463 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:58.308671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f3e103-c2e4-4368-9dd1-b1cc45790463 None admin] PATH_INFO: `/v3/projects/27a6d47620824971878ce596fab02fa4/users/d43e4a70df3c4919a2e14f9ad3265e10/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:58.310824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3e103-c2e4-4368-9dd1-b1cc45790463 None admin] RBAC: Authorizing `identity:create_grant(project_id=27a6d47620824971878ce596fab02fa4, user_id=d43e4a70df3c4919a2e14f9ad3265e10, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:58.312379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f3e103-c2e4-4368-9dd1-b1cc45790463 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:58.370760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 66/137] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:08:58 2022] PUT /identity/v3/projects/27a6d47620824971878ce596fab02fa4/users/d43e4a70df3c4919a2e14f9ad3265e10/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:08:59.227595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b2d4b-932a-4948-b543-6b7326af652d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:59.228095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b2d4b-932a-4948-b543-6b7326af652d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:59.228193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-924b2d4b-932a-4948-b543-6b7326af652d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:59.229582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 72/138] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:59.235522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333c3e4-0d9c-4a69-9afc-8e8879f1fa74 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:59.235983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333c3e4-0d9c-4a69-9afc-8e8879f1fa74 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:59.236367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3333c3e4-0d9c-4a69-9afc-8e8879f1fa74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:59.253775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3333c3e4-0d9c-4a69-9afc-8e8879f1fa74 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:59.255107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3333c3e4-0d9c-4a69-9afc-8e8879f1fa74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:59.330249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:59.346942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 67/139] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:08:59.390478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b2d4b-932a-4948-b543-6b7326af652d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:59.534404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-924b2d4b-932a-4948-b543-6b7326af652d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:59.578955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:59.579829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:59.580124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:59.580395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:59.600255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:08:59.608482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:08:59.609420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:08:59.610186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 73/140] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:08:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:08:59.612926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6f6f6-9e8f-4af7-bf43-7ed16b755316 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:59.613435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6f6f6-9e8f-4af7-bf43-7ed16b755316 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:59.613910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c6f6f6-9e8f-4af7-bf43-7ed16b755316 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:59.615495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 68/141] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:08:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:08:59.617210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:59.622227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:59.622623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:59.623009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:59.640945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:08:59.668794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:08:59.670089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:08:59.790555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046b5b34-9ba8-4264-a798-0919890044d6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:08:59.850249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 69/142] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:08:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:08:59.859096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:08:59.859763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:08:59.860043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:08:59.860316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:08:59.861097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:08:59.861403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:08:59.862781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:08:59.879248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 74/143] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:08:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:08:59.885204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:59.903023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:08:59.984081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:00.015969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a38174a-0937-4f6f-a18f-e6c330f20637 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:00.080184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daa06cc-adaa-4b3f-bbce-6b5e8ff36cd2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:00.101159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d2335a-2027-428b-9468-368d610ceaff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:00.102898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2335a-2027-428b-9468-368d610ceaff None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.104215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2335a-2027-428b-9468-368d610ceaff None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.105225 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d2335a-2027-428b-9468-368d610ceaff None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.108885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2335a-2027-428b-9468-368d610ceaff None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:00.110723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d2335a-2027-428b-9468-368d610ceaff None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:00.112948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 75/144] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:08:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:00.114352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:00.115713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c4d0b-afe6-4b7d-b07b-6342e308c093 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.116211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c4d0b-afe6-4b7d-b07b-6342e308c093 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.116746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-676c4d0b-afe6-4b7d-b07b-6342e308c093 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.118312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 76/145] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:00.124495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.125116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.134818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.140236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:00.141163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.141448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.141724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] PATH_INFO: `/v3/users/5acd460bee394e719cdafd0e39f18395` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.142939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] RBAC: Authorizing `identity:get_user(user_id=5acd460bee394e719cdafd0e39f18395)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:00.144413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:00.152038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 70/146] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:08:59 2022] GET /identity/v3/users/5acd460bee394e719cdafd0e39f18395 => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:00.153546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:00.155783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:00.156727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:00.233602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 77/147] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:00.240589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:00.342595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:00.373741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987b5d85-81dc-4ddc-b1ab-9f409584df06 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:00.377644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-165e8f25-4d2f-4787-b1c8-cc2f81196a75 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:00.417708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ed5007-6a3f-45ee-a838-3f25c8987056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:00.418714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5007-6a3f-45ee-a838-3f25c8987056 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.419198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5007-6a3f-45ee-a838-3f25c8987056 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.419705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ed5007-6a3f-45ee-a838-3f25c8987056 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.437293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:00.437982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.438279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.438551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.439276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:00.440776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:00.494922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed5007-6a3f-45ee-a838-3f25c8987056 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:00.496439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ed5007-6a3f-45ee-a838-3f25c8987056 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:00.565836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:00.595598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 71/148] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:00 2022] POST /identity/v3/users => generated 283 bytes in 436 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:00.595598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:00.604060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 78/149] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:00.605882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338afd48-08ef-4002-b7e3-9df47b16a1d5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.606182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338afd48-08ef-4002-b7e3-9df47b16a1d5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.606518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-338afd48-08ef-4002-b7e3-9df47b16a1d5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.609004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 79/150] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:00.618175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.618463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.618940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.647192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:00.648983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:00.684486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:00.772451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d18f73c-c19f-40a7-b63b-0fc5e27e9d8a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:00.800621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 80/151] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:00.800621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:00.828389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:00.829729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.830242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.830787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.852158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:00.860968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:00.861918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:00.862670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 72/152] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:00.863304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:00.913194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:00.919599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a40c4224-27e9-466c-9b2a-f321014c4fca None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:00.974179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:00.975067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:00.975392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:00.975624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:00.976960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:00.978522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:00.980456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 81/153] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:00 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:00.981162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:00.996244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209a7c2a-6cc2-411e-ab26-3a7d63b63c5a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.009019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:01.050011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.050880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.051170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.051719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.067322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:01.076020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:01.077118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:01.077873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:01.079544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 73/154] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:01.080452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.124723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32669d5c-cb61-44c3-abbb-8a8892a3a445 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.189557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.190894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.191385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.192053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.193163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:01.193678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.196133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.207219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 82/155] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:01.208542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.227061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9408de13-f40d-49c4-9914-1414b0d6812a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.253285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:01.274976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.275940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.276311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.276495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.277616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=5acd460bee394e719cdafd0e39f18395) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:01.277955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.279726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.288447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 74/156] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/role_assignments?user.id=5acd460bee394e719cdafd0e39f18395&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:01.289725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.332137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:01.347659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444df8ee-2162-4811-95b0-e561b2cefccc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.395565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.396392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.396813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.397100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.397965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:01.398434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.399919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07a506-91ae-4026-aba3-ae151ae16f82 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.410014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 83/157] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:01.411494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758fc23-eea5-4d15-985a-c0318c5bf758 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.411787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758fc23-eea5-4d15-985a-c0318c5bf758 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.412063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d758fc23-eea5-4d15-985a-c0318c5bf758 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 84/158] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:01.417803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758fc23-eea5-4d15-985a-c0318c5bf758 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.424583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09b5d1c-f75e-4004-9bd2-f08c4df163fe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.483007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d6da7e-1f5c-4b9e-a539-c88ccf88ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.485193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6da7e-1f5c-4b9e-a539-c88ccf88ddfe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.485501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6da7e-1f5c-4b9e-a539-c88ccf88ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.485786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d6da7e-1f5c-4b9e-a539-c88ccf88ddfe None admin] PATH_INFO: `/v3/users/19e4d96060764900a614e99e723b0bbf` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.488916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d6da7e-1f5c-4b9e-a539-c88ccf88ddfe None admin] RBAC: Authorizing `identity:get_user(user_id=19e4d96060764900a614e99e723b0bbf)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.491443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d6da7e-1f5c-4b9e-a539-c88ccf88ddfe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.497390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 75/159] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/users/19e4d96060764900a614e99e723b0bbf => generated 289 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:01.499042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.499333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.499622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.541863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:01.548481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:01.589933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:01.609877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d758fc23-eea5-4d15-985a-c0318c5bf758 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.666133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.667195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.667485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.667758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.669057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.670759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.672700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 85/160] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:01.674020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 76/161] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:01.675788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.682188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.719286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:01.811266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-face9ee3-81c8-4a20-983e-2659bda6a2c9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.827364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f5ad2a4-b30e-4c7b-b238-53a7c67b48c8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:01.855707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ee0b64-bdfd-4be6-bb98-a045abf678aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.856387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0b64-bdfd-4be6-bb98-a045abf678aa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.857291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0b64-bdfd-4be6-bb98-a045abf678aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.857585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ee0b64-bdfd-4be6-bb98-a045abf678aa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.875607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:01.876944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:01.877423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:01.877890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:01.879351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.881027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.882995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 86/162] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:01.884731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:01.935676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:01.946076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee0b64-bdfd-4be6-bb98-a045abf678aa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:01.947581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ee0b64-bdfd-4be6-bb98-a045abf678aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:01.989937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:02.026699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af7b8c64-a38a-46c1-97c6-e67a608e1ae4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:02.076885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d53e1a1-cf1a-4c45-820c-5ac75f65c2e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:02.078703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53e1a1-cf1a-4c45-820c-5ac75f65c2e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.080106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53e1a1-cf1a-4c45-820c-5ac75f65c2e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.080534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53e1a1-cf1a-4c45-820c-5ac75f65c2e0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.111208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d53e1a1-cf1a-4c45-820c-5ac75f65c2e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0d53e1a1-cf1a-4c45-820c-5ac75f65c2e0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:02.121720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:02.122676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:02.123432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:02.126279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 87/163] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:02.127877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dfa7-0c14-4486-848f-a267dd6b82c2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.128551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dfa7-0c14-4486-848f-a267dd6b82c2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.129054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2dfa7-0c14-4486-848f-a267dd6b82c2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.130808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 88/164] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:02.131098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 77/165] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:02.131954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2dfa7-0c14-4486-848f-a267dd6b82c2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.145211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.145425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.145627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.192250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:02.193403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:02.274879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:02.288417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2dfa7-0c14-4486-848f-a267dd6b82c2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:02.291949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 78/166] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:02.293529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.326753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076176c-615d-4953-babb-7859601293f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:02.327633 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076176c-615d-4953-babb-7859601293f8 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.327919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076176c-615d-4953-babb-7859601293f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.328198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1076176c-615d-4953-babb-7859601293f8 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e/users/19e4d96060764900a614e99e723b0bbf/roles/30181561a7e54d82b14a5ea28bafe082` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.330283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076176c-615d-4953-babb-7859601293f8 None admin] RBAC: Authorizing `identity:create_grant(project_id=66be157488cf4d29902de7af0526cf6e, user_id=19e4d96060764900a614e99e723b0bbf, role_id=30181561a7e54d82b14a5ea28bafe082)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:02.331736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1076176c-615d-4953-babb-7859601293f8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:02.381138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:02.425353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab33aa-6192-4054-b5ab-0681c01ff43a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:02.463116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 89/167] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:09:02 2022] PUT /identity/v3/projects/66be157488cf4d29902de7af0526cf6e/users/19e4d96060764900a614e99e723b0bbf/roles/30181561a7e54d82b14a5ea28bafe082 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:02.463895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076176c-615d-4953-babb-7859601293f8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.475171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:02.477084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.477714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.478252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.479847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:02.481669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:02.484111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 79/168] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:02 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:02.485419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.620091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05aa8e82-6fb2-4583-a00c-fac2cc8db14a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:02.678801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:02.689050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:02.690455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.690956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.691427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.704967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:02.718017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:02.718956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:02.719750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:02.727217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 80/169] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:02.728446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.789817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1076176c-615d-4953-babb-7859601293f8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:02.830941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:02.831387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.831622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.831898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.832682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:02.832990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:02.834396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:02.846524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 90/170] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:02.847397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.862339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:02.870509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51a263e-a182-4736-be9e-5ef4452bbb09 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:02.926328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:02.927375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:02.927861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:02.928335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:02.929627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=d43e4a70df3c4919a2e14f9ad3265e10) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:02.930174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:02.931912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:02.944063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 81/171] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:02 2022] GET /identity/v3/role_assignments?user.id=d43e4a70df3c4919a2e14f9ad3265e10&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:02.946012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:02.961440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b199a376-69c7-47e0-b13f-d91a07285145 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:03.006479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:03.008410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:03.008753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:03.009039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:03.009911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:03.010224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:03.011682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:03.031975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 91/172] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:03.037746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:03.051613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:03.124947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:03.157976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07026d51-f67e-4512-b76e-2d85628e0d13 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:03.164052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8e563-b59f-47a3-9bee-c1663f8e1df3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:03.206050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:03.206953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:03.207572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:03.207861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:03.209395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:03.211197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:03.213460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 92/173] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:03 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:03.220466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:03.253876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:03.254783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:03.255071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:03.255346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] PATH_INFO: `/v3/users/d43e4a70df3c4919a2e14f9ad3265e10` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:03.256638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] RBAC: Authorizing `identity:get_user(user_id=d43e4a70df3c4919a2e14f9ad3265e10)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:03.258214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:03.260346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 82/174] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:02 2022] GET /identity/v3/users/d43e4a70df3c4919a2e14f9ad3265e10 => generated 275 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:03.265431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:03.381143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f868667-5dc1-410f-8278-fad32a932de9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:03.435910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48f3812-bc1c-45c5-808c-f4fd81aab630 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:03.460621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:03.477118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94226c94-a843-493d-b83f-338c13549632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:03.477319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:03.484837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94226c94-a843-493d-b83f-338c13549632 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:03.484837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94226c94-a843-493d-b83f-338c13549632 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:03.485028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:03.485096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94226c94-a843-493d-b83f-338c13549632 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:03.485161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:03.485161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:03.499784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:03.593769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94226c94-a843-493d-b83f-338c13549632 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:03.595590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94226c94-a843-493d-b83f-338c13549632 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:03.624700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:03.626573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:03.826150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 93/175] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 605 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:03.830993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:03.865516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 83/176] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 601 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:03.872389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94226c94-a843-493d-b83f-338c13549632 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:03.912953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:03.918314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:04.076809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94226c94-a843-493d-b83f-338c13549632 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:04.097518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c27789-ae94-4dc9-b807-2d35b1491324 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:04.159074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:04.160305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.161003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.162640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] PATH_INFO: `/v3/projects/27a6d47620824971878ce596fab02fa4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.165360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] RBAC: Authorizing `identity:get_project(project_id=27a6d47620824971878ce596fab02fa4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:04.167671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:04.169834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 84/177] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:03 2022] GET /identity/v3/projects/27a6d47620824971878ce596fab02fa4 => generated 312 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:04.175984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:04.182082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:04.188598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.188598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.188598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.188598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:04.188598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:04.188598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 94/178] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:03 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:04.199959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:04.385406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:04.411995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74623723-e0d3-43e6-a7a3-ecad59bb577a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:04.416760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50db5af1-d980-4a73-91b0-82aae3e1a8de None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:04.436981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:04.483045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:04.483712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.483997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.484275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.485160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=27a6d47620824971878ce596fab02fa4, user.id=d43e4a70df3c4919a2e14f9ad3265e10) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:04.485477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:04.486942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4ccde-a7c9-47c8-9bad-2691ebc37aab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:04.497729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:04.498418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.498746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.499066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.499971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] RBAC: Adding query filter params (role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=d43e4a70df3c4919a2e14f9ad3265e10) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:04.500308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:04.510189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:04.512553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 85/179] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:04 2022] GET /identity/v3/role_assignments?user.id=d43e4a70df3c4919a2e14f9ad3265e10&scope.project.id=27a6d47620824971878ce596fab02fa4&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 608 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:04.514084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba016756-0c74-44bd-9b2e-c36892edf944 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.514374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba016756-0c74-44bd-9b2e-c36892edf944 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.514657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba016756-0c74-44bd-9b2e-c36892edf944 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.517383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 86/180] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:04.521536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 95/181] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:04 2022] GET /identity/v3/role_assignments?user.id=d43e4a70df3c4919a2e14f9ad3265e10&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=a7511c9c66e546dda3a737b069282dae => generated 608 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:04.541612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.541981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.542180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.563685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:04.565021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:04.662449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 87/182] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:04.711397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:04.857700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637f46d8-eaa3-43ed-ae8d-479676bb57f4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:04.896098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88401b6f-f264-4c5b-a289-205e5f78e1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:04.897330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88401b6f-f264-4c5b-a289-205e5f78e1d5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:04.898392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88401b6f-f264-4c5b-a289-205e5f78e1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:04.898942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88401b6f-f264-4c5b-a289-205e5f78e1d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:04.911789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88401b6f-f264-4c5b-a289-205e5f78e1d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-88401b6f-f264-4c5b-a289-205e5f78e1d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:04.927054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:04.927976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:04.929301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:04.931860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 96/183] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:04.937748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.003256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abb0b6-1630-4fd4-bea4-0c7f54e4bea6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.004343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abb0b6-1630-4fd4-bea4-0c7f54e4bea6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.005048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78abb0b6-1630-4fd4-bea4-0c7f54e4bea6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.008365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 97/184] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:05.018136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc25fb9-de11-4cc0-97b1-ada4780e41d6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.018921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc25fb9-de11-4cc0-97b1-ada4780e41d6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.019595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc25fb9-de11-4cc0-97b1-ada4780e41d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.045471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-acc25fb9-de11-4cc0-97b1-ada4780e41d6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:05.047123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-acc25fb9-de11-4cc0-97b1-ada4780e41d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:05.059038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.079550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.081217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4f7684-211d-4b02-8555-36ec529f08d2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.122232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.122921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.123211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.123490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.124253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:05.124584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.125994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.130768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 98/185] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:05.136199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb037-8adc-4227-9baa-1b676168a422 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.136199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb037-8adc-4227-9baa-1b676168a422 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.136199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476fb037-8adc-4227-9baa-1b676168a422 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.137143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 99/186] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:05.140273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 88/187] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:05.146099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.146434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.146754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.149324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.168387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:05.169632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:05.253166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 100/188] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:05.254065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.274044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08fcbc4e-c5ae-45df-b774-c40b92ec1e9c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.313766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.314691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.314973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.315278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] PATH_INFO: `/v3/users/b6e978cf8435405387d627802ddaf721` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.333296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] RBAC: Authorizing `identity:get_user(user_id=b6e978cf8435405387d627802ddaf721)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.334782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.341476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 89/189] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/users/b6e978cf8435405387d627802ddaf721 => generated 283 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:05.341476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.345380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.353331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.362116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731a2381-a8b5-44a8-aecb-f56c74e10cc6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.397425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.398872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.399219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.399548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.400908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.402478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.409297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 101/190] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:05.409297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.460666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3ccb58-b223-43eb-b184-504cb1e60486 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.505742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.507495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.507932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.508345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.524271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:05.525524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f968ca03-c8c9-4e77-87de-ba0b3151902f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:05.534317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:05.535226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:05.536007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:05.537152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 90/191] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:05.537519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.566812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.567480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.567768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.568062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.603096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.607736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.646535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.646799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-069449bf-1213-49b8-b725-fb5259a9782f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.648134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.680804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648e3189-4397-4a01-a179-b927e675c14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.681482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e3189-4397-4a01-a179-b927e675c14f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.681767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e3189-4397-4a01-a179-b927e675c14f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.682043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e3189-4397-4a01-a179-b927e675c14f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.682751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e3189-4397-4a01-a179-b927e675c14f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.684209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648e3189-4397-4a01-a179-b927e675c14f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.745851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 102/192] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:05.746631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.803125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.807332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 91/193] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:05 2022] POST /identity/v3/users => generated 297 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:05.808751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af640eb-25ce-4f96-96ea-6b4a2ae602c1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.809040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af640eb-25ce-4f96-96ea-6b4a2ae602c1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.809494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af640eb-25ce-4f96-96ea-6b4a2ae602c1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.810654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 92/194] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:05.811409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af640eb-25ce-4f96-96ea-6b4a2ae602c1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:05.876241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9ef911-6146-460b-9b2c-a772acbd99a0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.914136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.914833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.915117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.915401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.916172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:05.916482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.918031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d740ec6c-23f0-456c-931c-8de55c732278 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.933245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 103/195] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:05.933245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.933245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.933245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.950133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af640eb-25ce-4f96-96ea-6b4a2ae602c1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:05.950647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:05.953347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:05.971209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:05.993257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:05.994156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:05.994445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:05.994728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:05.996052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:05.997506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:05.999484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 93/196] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:06.001493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.033861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 104/197] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:06.034545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.148888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.217637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa50e0f-0611-412a-949c-2a5d1cb93f64 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:06.237700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06344324-53e7-44f3-95b0-85dbfb0c3efe None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:06.261596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:06.262276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:06.263065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:06.263405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:06.264613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=b6e978cf8435405387d627802ddaf721) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:06.264995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:06.266697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:06.274762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 105/198] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/role_assignments?user.id=b6e978cf8435405387d627802ddaf721&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:06.280723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.290656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:06.291844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:06.292317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:06.292809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] PATH_INFO: `/v3/users/b6e978cf8435405387d627802ddaf721` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:06.295773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] RBAC: Authorizing `identity:get_user(user_id=b6e978cf8435405387d627802ddaf721)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:06.297698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:06.299982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 94/199] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:05 2022] GET /identity/v3/users/b6e978cf8435405387d627802ddaf721 => generated 283 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:06.306969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.395987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dcb451-44f2-4bc9-81e8-6b67133b4570 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:06.416741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.421534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.442156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:06.443112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:06.443494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:06.443867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:06.460908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:06.472655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:06.478748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:06.479621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:06.480216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f46c8c-f528-43aa-ab74-191723f45549 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:06.480296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 106/200] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:06.480917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.522474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:06.524200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:06.524743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:06.525228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:06.597632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305cce78-f839-4123-bdb2-738552c34633 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:06.636710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:06.637398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:06.637681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:06.637956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:06.638716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:06.639020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:06.640426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:06.649909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 107/201] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:06.656786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.682687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:06.684172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:06.713025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.762297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c6fe39-4e34-461d-bca4-73006325592a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:06.781095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:06.808191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:06.809115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:06.809401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:06.809675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] PATH_INFO: `/v3/users/19e4d96060764900a614e99e723b0bbf` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:06.811894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] RBAC: Authorizing `identity:get_user(user_id=19e4d96060764900a614e99e723b0bbf)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:06.813349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:06.823518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 108/202] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/users/19e4d96060764900a614e99e723b0bbf => generated 289 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:06.831932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:06.861221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 95/203] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 555 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:06.873885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:07.057973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755e4bb2-ada9-430f-85da-ce2eff7b464a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:07.069011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:07.104301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c22702d-26bf-43f5-a510-353d2acac9bf None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:07.117020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:07.118329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:07.119009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:07.119431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:07.146051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:07.146917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:07.148063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:07.148240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:07.150576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:07.152062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:07.153823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 96/204] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:07.163584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:07.213375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:07.271691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:07.274137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:07.345358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3213dabb-4209-42ac-93ba-114e8bfe2ba0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:07.382150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:07.382924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:07.383216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:07.383495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:07.384361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] RBAC: Adding query filter params (role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=b6e978cf8435405387d627802ddaf721) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:07.384709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:07.386219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:07.400790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 97/205] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:07 2022] GET /identity/v3/role_assignments?user.id=b6e978cf8435405387d627802ddaf721&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=a7511c9c66e546dda3a737b069282dae => generated 259 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:07.435257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 109/206] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 604 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:07.440559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:07.557327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:07.574235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288ea149-04b0-41d9-b72e-a73c6b81e908 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:07.611507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38cbfdd4-ea88-4811-b06c-aaf0c5734dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:07.612490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbfdd4-ea88-4811-b06c-aaf0c5734dea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:07.612813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbfdd4-ea88-4811-b06c-aaf0c5734dea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:07.613096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38cbfdd4-ea88-4811-b06c-aaf0c5734dea None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:07.614400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbfdd4-ea88-4811-b06c-aaf0c5734dea None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:07.615877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38cbfdd4-ea88-4811-b06c-aaf0c5734dea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:07.617937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 98/207] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:07 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:07.623886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:07.637014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:07.783507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b6b7f7-b976-4fbf-9752-45536d44877b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:07.827253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:07.827923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:07.828258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:07.828542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:07.829455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] RBAC: Adding query filter params (role.id=30181561a7e54d82b14a5ea28bafe082, scope.project.id=66be157488cf4d29902de7af0526cf6e, user.id=19e4d96060764900a614e99e723b0bbf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:07.829769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:07.831236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b6f8aa-2e9e-4f8c-9f33-85862ee072fb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:07.838366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 110/208] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:07 2022] GET /identity/v3/role_assignments?user.id=19e4d96060764900a614e99e723b0bbf&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=30181561a7e54d82b14a5ea28bafe082 => generated 608 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:08.506106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eec01-5805-487d-a19e-b9a286a96895 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:08.506487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eec01-5805-487d-a19e-b9a286a96895 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:08.506663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66eec01-5805-487d-a19e-b9a286a96895 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:08.508342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 99/209] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:08.514187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37b98b-82cc-4787-985f-cf78b1fcf260 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:08.514496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37b98b-82cc-4787-985f-cf78b1fcf260 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:08.514748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da37b98b-82cc-4787-985f-cf78b1fcf260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:08.535803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-da37b98b-82cc-4787-985f-cf78b1fcf260 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:08.537054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-da37b98b-82cc-4787-985f-cf78b1fcf260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:08.611933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:08.617057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 111/210] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:08.644133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eec01-5805-487d-a19e-b9a286a96895 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:08.704772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cf04b-22a6-4931-b642-8a821078ba47 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:08.705717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cf04b-22a6-4931-b642-8a821078ba47 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:08.706222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30cf04b-22a6-4931-b642-8a821078ba47 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:08.707822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 112/211] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:08.713557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:08.714162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:08.714638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:08.733806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:08.735427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:08.755729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66eec01-5805-487d-a19e-b9a286a96895 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:08.791885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:08.792966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:08.793413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:08.793833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:08.795170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:08.796807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:08.798612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 100/212] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:08.804624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:08.812594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 113/213] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:08.830456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:08.839295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:08.910942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85f1d87-872a-4e7c-b272-c237f9a783f3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:08.960337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907af3e8-372d-41b4-be57-f33267564ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:08.960559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d82f0-f34c-4f37-a014-c4a100e7e70f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:08.961491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907af3e8-372d-41b4-be57-f33267564ee4 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:08.962298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907af3e8-372d-41b4-be57-f33267564ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:08.962738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907af3e8-372d-41b4-be57-f33267564ee4 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:08.964776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907af3e8-372d-41b4-be57-f33267564ee4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:08.966473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907af3e8-372d-41b4-be57-f33267564ee4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:08.998046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:09.003924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:09.004761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.005045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.005368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.016042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:09.025133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:09.026784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:09.027568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 114/214] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:09.032408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.101483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 101/215] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:08 2022] POST /identity/v3/users => generated 297 bytes in 298 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:09:09.148424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d91b6a8-fedd-4601-b38a-f4a7f7be96a5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:09.162595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5979-c2e8-4dae-9ed0-726b85da17ba None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.163660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5979-c2e8-4dae-9ed0-726b85da17ba None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.164371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786b5979-c2e8-4dae-9ed0-726b85da17ba None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.165928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 102/216] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:09.172064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.172359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.172685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.194613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:09.195340 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.195629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.195909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.197045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:09.197357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:09.198797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:09.200365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:09.201892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:09.210083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 115/217] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:09.217721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.288421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:09.301323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 103/218] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:09.326456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.337824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:09.347902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d733cda-f8c4-490c-8abd-6fa6f8d9b88b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:09.397650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:09.399535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.400039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.400384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] PATH_INFO: `/v3/users/042501b11bed47429b2912d75b89bb05` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.421216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] RBAC: Authorizing `identity:get_user(user_id=042501b11bed47429b2912d75b89bb05)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:09.424031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:09.426475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 116/219] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/users/042501b11bed47429b2912d75b89bb05 => generated 297 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:09.431949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.462562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6609552c-507d-41c7-b4ce-188b51e7c2e5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:09.509439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:09.512276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.512782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.513239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.528675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:09.537114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:09.538157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:09.539133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:09.541752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 104/220] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:09.547385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.570809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbe1530-062f-407c-a9b0-fba3317c6ae5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:09.582979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:09.611385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:09.612505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.613107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.613397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.648741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:09.689358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39242faa-6d38-4668-b53f-cb33145bba6e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:09.712221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:09.713814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:09.730440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:09.731130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:09.731413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:09.731691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:09.732452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:09.732792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:09.734202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:09.744417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 105/221] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:09.749940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.833343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 117/222] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:09.838285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:09.860958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:09.946257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f1dfe4-d484-4fa0-9a4e-dacba3309815 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:09.990937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc2f865-7773-408b-aba0-9fd05e06dd87 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.005301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:10.012334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57a1ea2-fce7-4352-b7af-3b1aa3cd469e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.021885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a1ea2-fce7-4352-b7af-3b1aa3cd469e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.022995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a1ea2-fce7-4352-b7af-3b1aa3cd469e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.023125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57a1ea2-fce7-4352-b7af-3b1aa3cd469e None admin] PATH_INFO: `/v3/users/b6e978cf8435405387d627802ddaf721` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.024681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a1ea2-fce7-4352-b7af-3b1aa3cd469e None admin] RBAC: Authorizing `identity:get_user(user_id=b6e978cf8435405387d627802ddaf721)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.026179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57a1ea2-fce7-4352-b7af-3b1aa3cd469e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.028505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 106/223] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/users/b6e978cf8435405387d627802ddaf721 => generated 283 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:10.029668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512455-5915-42f0-85be-d7a3e46ac498 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.029959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512455-5915-42f0-85be-d7a3e46ac498 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.030490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1512455-5915-42f0-85be-d7a3e46ac498 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.031772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 107/224] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:10.035427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b9063f-4f5f-4438-9383-b9e30bda9932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.036527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9063f-4f5f-4438-9383-b9e30bda9932 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.036872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9063f-4f5f-4438-9383-b9e30bda9932 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.037148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b9063f-4f5f-4438-9383-b9e30bda9932 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.038594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9063f-4f5f-4438-9383-b9e30bda9932 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.040037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b9063f-4f5f-4438-9383-b9e30bda9932 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.041560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1512455-5915-42f0-85be-d7a3e46ac498 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.042092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 118/225] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:09 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:10.043290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.043583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.043860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.062642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:10.063857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:10.140980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 119/226] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:10.141745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.160954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:10.162199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1512455-5915-42f0-85be-d7a3e46ac498 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.199200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.200006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.200293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.200589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.201836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.203311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.205080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 108/227] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:10 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:10.205859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.258131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d85f0c-a81f-47bd-a755-0ef53cfa8274 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.281940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:10.297645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.298319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.298605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.298882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.299729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=042501b11bed47429b2912d75b89bb05) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:10.300041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.301535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.310072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 120/228] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:10 2022] GET /identity/v3/role_assignments?user.id=042501b11bed47429b2912d75b89bb05&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:10.311121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.327259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb8b828-a9f8-4965-88d0-6bdfe266c065 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.404697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.407783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.408058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.408334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.429086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:10.436630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:10.458105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:10.459017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:10.459839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:10.464594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 109/229] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:10.466434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.499332 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da85310-ac35-42fd-ac16-1643591ba82a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.564757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.565740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.566026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.566319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/b6e978cf8435405387d627802ddaf721/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.569359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0c0feadab4f6894f27b847cbb48d1, user_id=b6e978cf8435405387d627802ddaf721, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.570884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.620771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b371c19-270a-454c-a48e-389e17561a3a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.650679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:10.659808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.660492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.660810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.661092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.661855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:10.662168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.664160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.674143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 110/230] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:10.681832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.689593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 121/231] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:09:10 2022] PUT /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/b6e978cf8435405387d627802ddaf721/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:10.827424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde4929-4704-4d53-a2eb-2df618fb191e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:10.864140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:10.865078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:10.865363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:10.865638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] PATH_INFO: `/v3/users/b6e978cf8435405387d627802ddaf721` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:10.867457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] RBAC: Authorizing `identity:get_user(user_id=b6e978cf8435405387d627802ddaf721)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:10.869041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:10.875326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 111/232] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:10 2022] GET /identity/v3/users/b6e978cf8435405387d627802ddaf721 => generated 283 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:10.876858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:10.994670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3007c0-b223-4fbd-a2ad-35ce48b3f0ea None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:11.025793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:11.031847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-578ac75a-1d4a-4ba5-863e-5aee3d6b2059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:11.032756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ac75a-1d4a-4ba5-863e-5aee3d6b2059 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:11.033043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ac75a-1d4a-4ba5-863e-5aee3d6b2059 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:11.033321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-578ac75a-1d4a-4ba5-863e-5aee3d6b2059 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:11.034511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ac75a-1d4a-4ba5-863e-5aee3d6b2059 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:11.035967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-578ac75a-1d4a-4ba5-863e-5aee3d6b2059 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:11.038044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 122/233] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:10 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:11.044269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:11.110367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:11.157523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3276353f-cb34-423b-8a5e-5e1fb8865d81 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:11.196069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992a904c-4b9b-4846-a5bc-7cb0f1dd7e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:11.197016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a904c-4b9b-4846-a5bc-7cb0f1dd7e3e None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:11.197304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a904c-4b9b-4846-a5bc-7cb0f1dd7e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:11.197587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992a904c-4b9b-4846-a5bc-7cb0f1dd7e3e None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/b6e978cf8435405387d627802ddaf721/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:11.199974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a904c-4b9b-4846-a5bc-7cb0f1dd7e3e None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0c0feadab4f6894f27b847cbb48d1, user_id=b6e978cf8435405387d627802ddaf721, role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:11.201482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992a904c-4b9b-4846-a5bc-7cb0f1dd7e3e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:11.316148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 112/234] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:09:11 2022] PUT /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/b6e978cf8435405387d627802ddaf721/roles/a7511c9c66e546dda3a737b069282dae => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:12.038487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068941f-4aba-439c-8d27-20c3236d167c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.039265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068941f-4aba-439c-8d27-20c3236d167c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.039265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7068941f-4aba-439c-8d27-20c3236d167c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.040728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 123/235] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:12.046377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.049018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.049018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.050045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a093efa-0000-4bf2-8f0a-4361db8cc814 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.051381 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a093efa-0000-4bf2-8f0a-4361db8cc814 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.051870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a093efa-0000-4bf2-8f0a-4361db8cc814 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.055382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 124/236] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:12 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:12.059311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.059833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.060558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.086537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:12.088183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:12.096015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:12.098619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:12.180628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 113/237] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:12.226459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:12.233372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 125/238] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:12.270018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:12.312154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:12.429432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61fed75-687e-4cf1-90ed-ad548ff51477 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:12.447893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d727a1-ed6e-4e7f-849c-ecb7b2f23319 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:12.467667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:12.474606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:12.476259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.483197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.483197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.483197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:12.483197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:12.483197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 126/239] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:12.487419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:12.495394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:12.496510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.497063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.497544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.511992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:12.520821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:12.522370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:12.523215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:12.525702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 114/240] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:12 2022] GET /identity/v3/roles/reader => generated 84 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:12.531786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:12.692709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d6baf-880a-4a4c-91d2-057f6930ca6c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:12.710767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d756d0-77a8-44d0-bfc3-02be76b7cf25 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:12.721049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:12.754682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:12.755365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.755651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.755927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.757189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:12.758802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:12.778775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:12.779463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.779744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.780269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.781074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:12.781382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:12.782804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:12.792238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 115/241] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:12 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:12.797998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:12.847101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 127/242] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:12 2022] POST /identity/v3/users => generated 293 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:12.931871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a458d4a-c41b-4d59-94a4-751d298d729a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:12.978755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d38648-c70f-45fb-b31b-40e27855c544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:12.979564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d38648-c70f-45fb-b31b-40e27855c544 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:12.979847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d38648-c70f-45fb-b31b-40e27855c544 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:12.980123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d38648-c70f-45fb-b31b-40e27855c544 None admin] PATH_INFO: `/v3/users/88ba7edddf614e3d8298a8a9c9f028c6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:12.997458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d38648-c70f-45fb-b31b-40e27855c544 None admin] RBAC: Authorizing `identity:get_user(user_id=88ba7edddf614e3d8298a8a9c9f028c6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:13.004819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d38648-c70f-45fb-b31b-40e27855c544 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:13.004819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 116/243] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:12 2022] GET /identity/v3/users/88ba7edddf614e3d8298a8a9c9f028c6 => generated 297 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:13.007555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.083246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:13.130854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef287070-5007-4d67-b8e0-951f6f1f9d0e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.131985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef287070-5007-4d67-b8e0-951f6f1f9d0e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.132129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef287070-5007-4d67-b8e0-951f6f1f9d0e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.134101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 117/244] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:13.140061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.140357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.140686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.170953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:13.172269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:13.200663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6d3e8d8-2025-43f9-92e9-5adb6d5b9028 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:13.217839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:13.250587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0337cfba-6286-484e-981a-1d07ca614cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:13.251565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337cfba-6286-484e-981a-1d07ca614cfc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.252744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337cfba-6286-484e-981a-1d07ca614cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.253043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0337cfba-6286-484e-981a-1d07ca614cfc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.282649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 118/245] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:13.312547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.339902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337cfba-6286-484e-981a-1d07ca614cfc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:13.341609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0337cfba-6286-484e-981a-1d07ca614cfc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:13.401168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:13.440372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3bed4-90ca-4ea2-957c-b9f99fd02c47 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:13.462782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 128/246] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:13.465638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f551c61-fa90-419e-9f62-0adb3a5dc3a9 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.466142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f551c61-fa90-419e-9f62-0adb3a5dc3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.466631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f551c61-fa90-419e-9f62-0adb3a5dc3a9 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.468159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 129/247] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:13.473159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f551c61-fa90-419e-9f62-0adb3a5dc3a9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.488837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46796851-1cf8-4a73-b9f9-d8741c1d9f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:13.489656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46796851-1cf8-4a73-b9f9-d8741c1d9f05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.489944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46796851-1cf8-4a73-b9f9-d8741c1d9f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.490220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46796851-1cf8-4a73-b9f9-d8741c1d9f05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.501678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46796851-1cf8-4a73-b9f9-d8741c1d9f05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-46796851-1cf8-4a73-b9f9-d8741c1d9f05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:13.510079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:13.510986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:13.512184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 119/248] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:13.512899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.516986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.517281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.540968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:13.542781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:13.558596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:13.606401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f551c61-fa90-419e-9f62-0adb3a5dc3a9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:13.631193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 120/249] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:13.632937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.665933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:13.666828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.667113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.667396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.669349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:13.670787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:13.674472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 130/250] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:13.675503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.703670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:13.757552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21be98ed-2784-4743-844c-cbc30741d6b0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:13.787380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a44b70-894e-4d5d-9248-829049d970dd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:13.802361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:13.803734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.804308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.804888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.805812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:13.806432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:13.808332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:13.820545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 121/251] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:13.821666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.829991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:13.830948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.831333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.831709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.843107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:13.851725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:13.852870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:13.853853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:13.855875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 131/252] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:13.857870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:13.866539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:13.941869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef9efce9-a87a-444d-afb4-7d5c1fbea392 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:13.982202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:13.983470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:13.983999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:13.984511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:13.985827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] RBAC: Adding query filter params (role.id=30181561a7e54d82b14a5ea28bafe082, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=88ba7edddf614e3d8298a8a9c9f028c6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:13.986368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:13.988107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:13.996341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 122/253] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/role_assignments?user.id=88ba7edddf614e3d8298a8a9c9f028c6&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=30181561a7e54d82b14a5ea28bafe082 => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:13.997572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:14.068849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:14.097012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:14.117929 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86a6df6-6f3e-4c25-8b9f-3c2b371f1eb4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:14.127520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a5fa7-cdb5-46b1-8676-53ff7808fb38 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:14.169435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4819b01-556a-4145-bb84-158873830287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:14.171387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4819b01-556a-4145-bb84-158873830287 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.171915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4819b01-556a-4145-bb84-158873830287 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.172402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4819b01-556a-4145-bb84-158873830287 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.174733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4819b01-556a-4145-bb84-158873830287 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:14.175332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4819b01-556a-4145-bb84-158873830287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:14.177097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4819b01-556a-4145-bb84-158873830287 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:14.188883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 123/254] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:14.197915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4819b01-556a-4145-bb84-158873830287 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:14.247002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:14.247915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.248200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.248475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] PATH_INFO: `/v3/users/042501b11bed47429b2912d75b89bb05` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.257548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] RBAC: Authorizing `identity:get_user(user_id=042501b11bed47429b2912d75b89bb05)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:14.258895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:14.260729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 132/255] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:13 2022] GET /identity/v3/users/042501b11bed47429b2912d75b89bb05 => generated 297 bytes in 404 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:14.267032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:14.342476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4819b01-556a-4145-bb84-158873830287 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:14.410369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:14.411279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.411567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.411845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] PATH_INFO: `/v3/users/b6e978cf8435405387d627802ddaf721` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.414081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] RBAC: Authorizing `identity:get_user(user_id=b6e978cf8435405387d627802ddaf721)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:14.415570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:14.417500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 124/256] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:14 2022] GET /identity/v3/users/b6e978cf8435405387d627802ddaf721 => generated 283 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:14.422704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:14.432921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:14.485237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455cbab7-d819-488f-a5bd-3acfd77c3761 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:14.536297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca0357-dcc0-490e-846f-d0c4b61b01d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:14.541381 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0357-dcc0-490e-846f-d0c4b61b01d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.541672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0357-dcc0-490e-846f-d0c4b61b01d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.541952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca0357-dcc0-490e-846f-d0c4b61b01d1 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.544316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca0357-dcc0-490e-846f-d0c4b61b01d1 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:14.545803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ca0357-dcc0-490e-846f-d0c4b61b01d1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:14.548617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 133/257] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:14 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:14.563218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23547d1-98c7-44ea-9566-d1c794129079 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.563515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23547d1-98c7-44ea-9566-d1c794129079 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.563773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23547d1-98c7-44ea-9566-d1c794129079 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.565461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 134/258] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:14.566166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23547d1-98c7-44ea-9566-d1c794129079 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:14.598001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:14.631894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc634b1f-6552-44b3-a639-b6fbba74e607 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:14.678571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:14.679544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.679544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.679861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.732976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b23547d1-98c7-44ea-9566-d1c794129079 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:14.806117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ee2f18-b687-44eb-9569-bdfb05e3ff57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:14.807061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee2f18-b687-44eb-9569-bdfb05e3ff57 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.807345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee2f18-b687-44eb-9569-bdfb05e3ff57 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.807622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee2f18-b687-44eb-9569-bdfb05e3ff57 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/042501b11bed47429b2912d75b89bb05/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.815707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee2f18-b687-44eb-9569-bdfb05e3ff57 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0c0feadab4f6894f27b847cbb48d1, user_id=042501b11bed47429b2912d75b89bb05, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:14.818386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ee2f18-b687-44eb-9569-bdfb05e3ff57 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:14.823514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:14.825988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:14.828674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:14.933150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 135/259] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:09:14 2022] PUT /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/042501b11bed47429b2912d75b89bb05/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:14.934441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:14.934779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:14.935060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:14.955663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 125/260] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:14.962524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:14.988675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:14.989910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:15.182350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 136/261] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:15.200626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc74527-2589-4dc5-a3ae-0282944e4ab0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:15.203177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:15.226358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:15.238393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:15.264771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:15.267488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:15.267488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:15.267488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:15.267823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:15.269326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:15.274523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 126/262] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:14 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:15.281708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:15.430191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d756dbe-bc50-4a41-9a90-34530e65d859 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:15.451770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e589aacc-4418-42b2-910b-afc5d8b92ec6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:15.487023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:15.487872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:15.488152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:15.488424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:15.489297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=b6e978cf8435405387d627802ddaf721) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:15.489607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:15.491630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:15.499363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 127/263] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:15 2022] GET /identity/v3/role_assignments?user.id=b6e978cf8435405387d627802ddaf721&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:15.539689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:15.540917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:15.541435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:15.541924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:15.569436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:15.577861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:15.578815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:15.579592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 137/264] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 353 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:15.585731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:15.644852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:15.735219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70920693-2672-4236-93d6-1f92628e04ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:15.779585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:15.780524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:15.780830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:15.781105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:15.781954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:15.782260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:15.783645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:15.796139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 128/265] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:15.801785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:15.898112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:15.938231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e62ee03-a0d0-4668-83a3-d2a526d315f6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:15.981461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:15.982584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:15.982954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:15.983238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] PATH_INFO: `/v3/users/b6e978cf8435405387d627802ddaf721` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:15.989401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] RBAC: Authorizing `identity:get_user(user_id=b6e978cf8435405387d627802ddaf721)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:15.990912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:15.992823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 138/266] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:15 2022] GET /identity/v3/users/b6e978cf8435405387d627802ddaf721 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:15.998466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:16.129917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:16.136387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456fb651-3f4e-49cf-854a-3b0136c01093 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:16.178265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc780875-e5ec-4e90-9b1c-400e442b7930 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:16.179312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc780875-e5ec-4e90-9b1c-400e442b7930 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.179825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc780875-e5ec-4e90-9b1c-400e442b7930 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.180309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc780875-e5ec-4e90-9b1c-400e442b7930 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.301205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc780875-e5ec-4e90-9b1c-400e442b7930 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:16.302639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc780875-e5ec-4e90-9b1c-400e442b7930 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:16.422413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:16.432901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 129/267] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 435 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:16.435676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:16.459171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50d8ee-5fa6-4abf-8056-8129fbe16365 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.459950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50d8ee-5fa6-4abf-8056-8129fbe16365 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.460030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad50d8ee-5fa6-4abf-8056-8129fbe16365 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.461677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 130/268] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:16.467141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.467429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.467697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.497801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:16.499449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:16.583346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd7ee7-7f08-40c1-93ef-cf33996a82ba None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:16.623675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:16.624604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.624893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.625171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.626522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:16.628034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:16.630190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 139/269] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:16 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:16.636197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:16.640151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:16.710392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 131/270] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:16.750476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e22e6d-080c-4116-abba-63ea82034b58 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:16.761668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:16.790684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:16.791774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.792285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.792802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.793855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] RBAC: Adding query filter params (role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=b6e978cf8435405387d627802ddaf721) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:16.794378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:16.796015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf932b2b-caa3-4c3e-a822-39d6e4969106 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:16.803112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 140/271] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:16 2022] GET /identity/v3/role_assignments?user.id=b6e978cf8435405387d627802ddaf721&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=a7511c9c66e546dda3a737b069282dae => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:16.804327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b52f0-b0e2-45bb-ae94-69bc96d61943 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.804645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b52f0-b0e2-45bb-ae94-69bc96d61943 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.804930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691b52f0-b0e2-45bb-ae94-69bc96d61943 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.806508 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 141/272] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:16.812200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783bfa3-ebc8-4efa-94c3-0c0197ba5afa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.812493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783bfa3-ebc8-4efa-94c3-0c0197ba5afa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.812816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783bfa3-ebc8-4efa-94c3-0c0197ba5afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.842119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d783bfa3-ebc8-4efa-94c3-0c0197ba5afa None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:16.843311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d783bfa3-ebc8-4efa-94c3-0c0197ba5afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:16.917858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d5a2741-9c83-4448-9690-77b5b54fe0ff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:16.940672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:16.948254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 142/273] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:16.950092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c1345-aec1-4748-991c-ad45db501c3f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.950384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c1345-aec1-4748-991c-ad45db501c3f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.950977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185c1345-aec1-4748-991c-ad45db501c3f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.952991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 143/274] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:16.964596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:16.964806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.964806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.964806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.977040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:16.977904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:16.978168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:16.978449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:16.995254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:17.003691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:17.005169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 132/275] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:16 2022] GET /identity/v3/roles/member => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:17.007028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.049927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:17.051571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:17.165398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 144/276] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:17.166268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.203990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5980fb3a-af6b-40f4-950a-21fcf916c423 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.250714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.251623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.252004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.252342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.253640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:17.255272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:17.258012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 133/277] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:17.258707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.311559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608f1bc7-b618-4e6d-8ac3-9501ca52f3f4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.331983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:17.352650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:17.362243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.363213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.363505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.363789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.366087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:17.366087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:17.366833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:17.376012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 145/278] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:17.377441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.394349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28700950-895a-4940-a9a2-2eaaf906207d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.436536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.437289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.437574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.437850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.449351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:17.457152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:17.458070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 134/279] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/roles/reader => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:17.464081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.513059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a4ff2-bd05-4507-a505-738dc3dd12a2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.567339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.568015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.568330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.568626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.569344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:17.571234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:17.590887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69fa887-fe10-4f21-b9b9-8175a9f05d77 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.606329 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:17.630178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.631841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.634197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.634197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] PATH_INFO: `/v3/users/1e6a18fb2f9142b19f7e4a7cb3c710c3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.649950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] RBAC: Authorizing `identity:get_user(user_id=1e6a18fb2f9142b19f7e4a7cb3c710c3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:17.651523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:17.653696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 135/280] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/users/1e6a18fb2f9142b19f7e4a7cb3c710c3 => generated 293 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:17.657473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.697327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 146/281] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:17 2022] POST /identity/v3/users => generated 293 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:17.697327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.766971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:17.795773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39571d0-9a64-4abc-8070-600dd6ea3884 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.837043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a63f5e-64cd-4650-b92f-36b40b7acb4b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:17.837588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.838296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.838639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.838920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.839859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:17.840054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:17.841860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:17.851698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 136/282] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:17.858375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:17.894492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:17.895174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.895457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.895734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.896726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, user.id=1e6a18fb2f9142b19f7e4a7cb3c710c3, scope.system=all) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:17.897006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:17.898447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f19aa-2784-4d27-ab76-ee8bfd1f2298 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:17.908859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 147/283] 10.222.0.70 () {56 vars in 1308 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/role_assignments?user.id=1e6a18fb2f9142b19f7e4a7cb3c710c3&scope.system=all&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 226 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:17.932547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:17.934878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93c8a3-3226-4577-8cd7-69f5c0bb7e37 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.935177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93c8a3-3226-4577-8cd7-69f5c0bb7e37 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.936030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c93c8a3-3226-4577-8cd7-69f5c0bb7e37 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.938092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 148/284] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:17.944892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:17.945440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:17.945873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:17.968892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:17.970537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:18.145542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a8454e-8c64-4b56-b675-406c3de689ce None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:18.188106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:18.189034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:18.192592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:18.192592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] PATH_INFO: `/v3/users/88ba7edddf614e3d8298a8a9c9f028c6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:18.192592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] RBAC: Authorizing `identity:get_user(user_id=88ba7edddf614e3d8298a8a9c9f028c6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:18.192853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:18.195176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 137/285] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:17 2022] GET /identity/v3/users/88ba7edddf614e3d8298a8a9c9f028c6 => generated 297 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:18.200246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:18.218070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 149/286] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 275 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:18.297964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:18.350819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:18.386974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718e4bfa-9c86-439f-8f3c-e066d7a78dc2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:18.389477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:18.425275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c53814d-f19b-4596-832a-abf43fa3006d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:18.467489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:18.467698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:18.468366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:18.468539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:18.468694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:18.468959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:18.469053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:18.469349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:18.470653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:18.472190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:18.480470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:18.481040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 138/287] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:18 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:18.486646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:18.496894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:18.497825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:18.498682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 150/288] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:18 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:18.518174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:18.621126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fff2f7-f488-466f-b58f-c583a6ea6dca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:18.661404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:18.662608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:18.662982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:18.663259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/88ba7edddf614e3d8298a8a9c9f028c6/roles/30181561a7e54d82b14a5ea28bafe082` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:18.665341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0c0feadab4f6894f27b847cbb48d1, user_id=88ba7edddf614e3d8298a8a9c9f028c6, role_id=30181561a7e54d82b14a5ea28bafe082)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:18.666799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:18.718533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e933d47-d2e3-4bdd-a85a-bea56a98c9fc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:18.737327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:18.752870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 139/289] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:09:18 2022] PUT /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/users/88ba7edddf614e3d8298a8a9c9f028c6/roles/30181561a7e54d82b14a5ea28bafe082 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:09:18.758293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:18.759013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:18.759354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:18.759643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:18.760494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:18.760837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:18.762451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:18.775183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 151/290] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:18 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:18.781767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:18.843323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:18.954114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eee6d10-16df-4452-a955-e04d5c0a4893 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:19.006304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:19.007199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:19.007536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:19.007812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] PATH_INFO: `/v3/users/042501b11bed47429b2912d75b89bb05` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:19.009078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] RBAC: Authorizing `identity:get_user(user_id=042501b11bed47429b2912d75b89bb05)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:19.010527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:19.012154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 140/291] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:18 2022] GET /identity/v3/users/042501b11bed47429b2912d75b89bb05 => generated 297 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:19.017675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:19.138349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca24b7d3-f14b-4616-89cd-9c87c0ba4f72 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:19.194505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:19.195182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:19.195468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:19.195747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:19.229068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:19.268616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:19.270090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:19.374872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 152/292] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:19.380264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:19.520235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:19.528661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0df7-f613-4e51-9276-59214941d519 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:19.576023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df13a24-85c0-4c6e-888b-208aa542866b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:19.576955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13a24-85c0-4c6e-888b-208aa542866b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:19.577238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13a24-85c0-4c6e-888b-208aa542866b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:19.577513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df13a24-85c0-4c6e-888b-208aa542866b None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:19.578731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df13a24-85c0-4c6e-888b-208aa542866b None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:19.580187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df13a24-85c0-4c6e-888b-208aa542866b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:19.581930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 141/293] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:19 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:19.588294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:19.718018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:19.767689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6e3d34-f441-4be7-b8dc-e24d902cec0d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:19.812264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:19.812895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:19.813180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:19.813461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:19.814297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=042501b11bed47429b2912d75b89bb05) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:19.814615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:19.816053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d450e49-70b4-47c1-80db-76b68fd07e3a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:19.823368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 153/294] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:19 2022] GET /identity/v3/role_assignments?user.id=042501b11bed47429b2912d75b89bb05&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 608 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:20.596392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991424-02e0-45be-a191-e2e5a7ee95d1 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.597396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991424-02e0-45be-a191-e2e5a7ee95d1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.597913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96991424-02e0-45be-a191-e2e5a7ee95d1 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.599902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 142/295] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:20.605746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.606199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.606385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.628732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:20.629922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:20.676418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08881d83-3a65-43a1-88e9-e32575ac58bc None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.676803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08881d83-3a65-43a1-88e9-e32575ac58bc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.677081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08881d83-3a65-43a1-88e9-e32575ac58bc None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.678986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 143/296] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:20.684327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec02dc-f432-4905-a7fe-68f69e81191a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.684640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec02dc-f432-4905-a7fe-68f69e81191a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.684920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dec02dc-f432-4905-a7fe-68f69e81191a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.703977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6dec02dc-f432-4905-a7fe-68f69e81191a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:20.705531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6dec02dc-f432-4905-a7fe-68f69e81191a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:20.716111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 154/297] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:20.742908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:20.789656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 144/298] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:20.798185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea6ace-64dc-49a3-896b-d9aab1f1bb4d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.798479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea6ace-64dc-49a3-896b-d9aab1f1bb4d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.798764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dea6ace-64dc-49a3-896b-d9aab1f1bb4d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.804276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 145/299] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:20.805570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.805857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.806131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.809192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:20.832450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:20.834238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:20.860595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cb4615-efa3-4812-9fb9-e84d6777827d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:20.899201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:20.899983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:20.900278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:20.900538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:20.912615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 146/300] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:20.912615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:20.920780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:20.930807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:20.941202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 155/301] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:20 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:20.942002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:20.942709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.056224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5699f5f3-a92b-4918-a99a-7b931b55da7a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.079428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936dfc4e-cdc6-4e0f-ae4b-394aa3255385 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.096230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.097110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.097392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.097664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.099126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.100718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.101129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.102493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 147/302] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:21.105619 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.121220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.121951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.122238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.122516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.135191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:21.142858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:21.143723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:21.144611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 156/303] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:21.145272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.231999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.239382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4459f1-929b-4166-a625-25729dcdd5da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.258249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b512356-3e7a-41f5-880e-6089197bc52e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.278511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.279181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.279713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.279983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.281204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:21.281517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.282926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.291282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 148/304] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:21.292080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.299480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952bca78-5cbd-4c67-9839-dffa010828ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.300155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bca78-5cbd-4c67-9839-dffa010828ff None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.300436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bca78-5cbd-4c67-9839-dffa010828ff None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.300757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952bca78-5cbd-4c67-9839-dffa010828ff None admin] PATH_INFO: `/v3/groups` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.301461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bca78-5cbd-4c67-9839-dffa010828ff None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.303018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952bca78-5cbd-4c67-9839-dffa010828ff None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.394631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 157/305] 10.222.0.70 () {60 vars in 1134 bytes} [Mon May 23 13:09:21 2022] POST /identity/v3/groups => generated 229 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:21.396143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41847f6-166d-4929-b0d0-90fd2d0cf8a7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.396682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41847f6-166d-4929-b0d0-90fd2d0cf8a7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.397303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41847f6-166d-4929-b0d0-90fd2d0cf8a7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.398520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 158/306] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:21.399417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41847f6-166d-4929-b0d0-90fd2d0cf8a7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.400549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a95c75a0-13b4-4a3b-977c-0aebf5a6b40d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.448767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.449462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.449749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.450028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.450842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:21.451117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.452548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d0bb0e-1665-4ff6-9b8c-f268aac8aa2d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.458906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.467927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 149/307] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:21.467927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.467927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.467927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.483018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:21.484266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:21.501648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.545333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41847f6-166d-4929-b0d0-90fd2d0cf8a7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.566542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 150/308] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:21.572817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.603244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.604130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.604413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.604791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] PATH_INFO: `/v3/users/a67591059b1c46a597f1eee7e20862fa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.622799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] RBAC: Authorizing `identity:get_user(user_id=a67591059b1c46a597f1eee7e20862fa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.625342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.626957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 159/309] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/users/a67591059b1c46a597f1eee7e20862fa => generated 293 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:21.629238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.685025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97941271-8e34-46a1-a640-82f44a19e5a4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.724041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.725532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.725889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.726163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] PATH_INFO: `/v3/users/1e6a18fb2f9142b19f7e4a7cb3c710c3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.727412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] RBAC: Authorizing `identity:get_user(user_id=1e6a18fb2f9142b19f7e4a7cb3c710c3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.728914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.736861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 151/310] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/users/1e6a18fb2f9142b19f7e4a7cb3c710c3 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:21.737047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.737138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.745407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d96674-60a7-49dd-910b-341480884d5f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.757762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:21.794767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.795585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.795896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.796204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.811844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:21.820418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:21.821349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 160/311] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:21.822957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.856160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a686cd-8394-43f7-86ee-ce22720d00ac None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.891267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.891873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.892155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.892428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.893365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] RBAC: Adding query filter params (role.id=30181561a7e54d82b14a5ea28bafe082, user.id=a67591059b1c46a597f1eee7e20862fa, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:21.893675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.895113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:21.901169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 152/312] 10.222.0.70 () {56 vars in 1308 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/role_assignments?user.id=a67591059b1c46a597f1eee7e20862fa&scope.system=all&role.id=30181561a7e54d82b14a5ea28bafe082 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:21.901820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:21.945239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3747b3d4-39c5-4d88-a79c-0913c1564594 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:21.988781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:21.990199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:21.990831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:21.991347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] PATH_INFO: `/v3/system/users/1e6a18fb2f9142b19f7e4a7cb3c710c3/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:21.993241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1e6a18fb2f9142b19f7e4a7cb3c710c3, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:21.994888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:22.012708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:22.034213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4467b6d-0711-4bef-8704-aa509420a4a5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:22.088339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:22.089098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:22.089388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:22.089668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:22.090455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:22.090791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:22.094878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:22.104647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 161/313] 10.222.0.70 () {58 vars in 1334 bytes} [Mon May 23 13:09:21 2022] PUT /identity/v3/system/users/1e6a18fb2f9142b19f7e4a7cb3c710c3/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:22.106573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 153/314] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:22.115490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:22.141040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:22.237370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5569dd-63df-4fe7-b5a9-06b5aa8afbb6 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:22.283398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:22.284719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:22.285209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:22.285709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] PATH_INFO: `/v3/users/88ba7edddf614e3d8298a8a9c9f028c6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:22.287475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] RBAC: Authorizing `identity:get_user(user_id=88ba7edddf614e3d8298a8a9c9f028c6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:22.289307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:22.296049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 162/315] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:22 2022] GET /identity/v3/users/88ba7edddf614e3d8298a8a9c9f028c6 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:22.301581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:22.408743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc095ca-0ba2-42b6-ba95-8c5e79f76271 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:22.446545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:22.447636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:22.448215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:22.448725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:22.466448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:22.529125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:22.530778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:22.640681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 154/316] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:22.642246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:22.737281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:22.753709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25074fc1-1ad4-4d00-a50f-c89e0e59a6ab None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:22.789322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7f701d-6a9e-48f6-813b-7eb7bdff75f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:22.790149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f701d-6a9e-48f6-813b-7eb7bdff75f1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:22.790434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f701d-6a9e-48f6-813b-7eb7bdff75f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:22.790708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7f701d-6a9e-48f6-813b-7eb7bdff75f1 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:22.791916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f701d-6a9e-48f6-813b-7eb7bdff75f1 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:22.793458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7f701d-6a9e-48f6-813b-7eb7bdff75f1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:22.795577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 163/317] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:22 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:22.801558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:22.857142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:22.915234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d69804-3c7e-431e-8c5b-b4048ad22dbb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:22.949651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:22.950325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:22.950615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:22.950895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:22.951783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] RBAC: Adding query filter params (role.id=30181561a7e54d82b14a5ea28bafe082, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1, user.id=88ba7edddf614e3d8298a8a9c9f028c6) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:22.952106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:22.953584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70522a18-9c97-4c23-9bd2-faedd2aaea50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:22.960815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 155/318] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:22 2022] GET /identity/v3/role_assignments?user.id=88ba7edddf614e3d8298a8a9c9f028c6&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=30181561a7e54d82b14a5ea28bafe082 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:23.752390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314871c-7739-4d40-a90b-b218cedd27d1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:23.752940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314871c-7739-4d40-a90b-b218cedd27d1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:23.752940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314871c-7739-4d40-a90b-b218cedd27d1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:23.754339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 164/319] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:23.759991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:23.760285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:23.760581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:23.777975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:23.779167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:23.847710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 156/320] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:23.870663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314871c-7739-4d40-a90b-b218cedd27d1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:23.978761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314871c-7739-4d40-a90b-b218cedd27d1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:24.021005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d94474-6831-428d-ad90-52d7386ef53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:24.022123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94474-6831-428d-ad90-52d7386ef53f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.022413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94474-6831-428d-ad90-52d7386ef53f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.022690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d94474-6831-428d-ad90-52d7386ef53f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.023987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d94474-6831-428d-ad90-52d7386ef53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:24.025495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d94474-6831-428d-ad90-52d7386ef53f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:24.032200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 165/321] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:23 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:24.033553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:24.072542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:24.147333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51122f14-71ad-4707-9111-a6fafdd93dd5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:24.193198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7487e84-75cc-40ed-ab9e-6f3c94cd74b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:24.194065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487e84-75cc-40ed-ab9e-6f3c94cd74b7 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.194351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487e84-75cc-40ed-ab9e-6f3c94cd74b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.194626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7487e84-75cc-40ed-ab9e-6f3c94cd74b7 None admin] PATH_INFO: `/v3/groups` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.195338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7487e84-75cc-40ed-ab9e-6f3c94cd74b7 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:24.196899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7487e84-75cc-40ed-ab9e-6f3c94cd74b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:24.288287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 157/322] 10.222.0.70 () {60 vars in 1134 bytes} [Mon May 23 13:09:24 2022] POST /identity/v3/groups => generated 226 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:24.290467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac801-40b4-412e-836a-3245e3c6a6a7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.294041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac801-40b4-412e-836a-3245e3c6a6a7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.294041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499ac801-40b4-412e-836a-3245e3c6a6a7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.294041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 166/323] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:24.298470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.298771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.299047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.318478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:24.319764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:24.390912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a070935-b089-4ff4-ac3a-5bab14241e64 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.391241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a070935-b089-4ff4-ac3a-5bab14241e64 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.391470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a070935-b089-4ff4-ac3a-5bab14241e64 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.392948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 167/324] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:24.397743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 158/325] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:24.399514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.399808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.400084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.418172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:24.419395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:24.421356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:24.441593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:24.451578 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:24.503599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 168/326] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:24.527034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:24.535516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5026ef9e-e2c7-4e7b-96bc-72a70c5ed70b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:24.581217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:24.582381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.582819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.583757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.600482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:24.608778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:24.609680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:24.610457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:24.612507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 159/327] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:24 2022] GET /identity/v3/roles/reader => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:24.619295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:24.655733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3256ab9f-079f-463e-b598-6e269398b182 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:24.702758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:24.703580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.703865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.704143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.722109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:24.731138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:24.732076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:24.733564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 169/328] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:24.742242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:24.752241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:24.764500 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:24.771158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bb8a16-0363-4b34-a0d7-5324a6dca24e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:24.821376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:24.822443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.822932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.823422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.824390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:24.825609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:24.828094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:24.839001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 160/329] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:24 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:24.851044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:24.905576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae88033-edaf-434d-8ab5-83289bd1b024 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:24.955468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:24.955468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:24.955765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:24.956047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:24.956688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] RBAC: Adding query filter params (name=member) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:24.956990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:24.958420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:24.978459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 170/330] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:24.983609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:25.008201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb1b649-9ae4-4593-910c-e97d1b0ee763 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:25.075845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:25.076908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:25.077262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:25.077605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] PATH_INFO: `/v3/users/a67591059b1c46a597f1eee7e20862fa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:25.079605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] RBAC: Authorizing `identity:get_user(user_id=a67591059b1c46a597f1eee7e20862fa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:25.081216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:25.083083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 161/331] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:24 2022] GET /identity/v3/users/a67591059b1c46a597f1eee7e20862fa => generated 293 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:25.090398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:25.113088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:25.133476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:25.169491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b00e3d-aa8a-4da1-9f71-4d2f987e1e52 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:25.221588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:25.223666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:25.224720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:25.225191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] PATH_INFO: `/v3/users/1e6a18fb2f9142b19f7e4a7cb3c710c3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:25.227075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] RBAC: Authorizing `identity:get_user(user_id=1e6a18fb2f9142b19f7e4a7cb3c710c3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:25.229802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:25.231935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 171/332] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:24 2022] GET /identity/v3/users/1e6a18fb2f9142b19f7e4a7cb3c710c3 => generated 293 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:25.238139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:25.243436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77da2188-7c9b-474e-84a6-6418b80b3b99 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:25.286045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015a62b6-7e04-47c0-8b00-a5823d93111e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:25.287018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a62b6-7e04-47c0-8b00-a5823d93111e None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:25.287302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a62b6-7e04-47c0-8b00-a5823d93111e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:25.287576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015a62b6-7e04-47c0-8b00-a5823d93111e None admin] PATH_INFO: `/v3/system/users/a67591059b1c46a597f1eee7e20862fa/roles/30181561a7e54d82b14a5ea28bafe082` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:25.289872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a62b6-7e04-47c0-8b00-a5823d93111e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=a67591059b1c46a597f1eee7e20862fa, role_id=30181561a7e54d82b14a5ea28bafe082)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:25.290533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015a62b6-7e04-47c0-8b00-a5823d93111e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:25.361979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29288280-433f-4eaa-9924-795c48449e76 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:25.373737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 162/333] 10.222.0.70 () {58 vars in 1334 bytes} [Mon May 23 13:09:25 2022] PUT /identity/v3/system/users/a67591059b1c46a597f1eee7e20862fa/roles/30181561a7e54d82b14a5ea28bafe082 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:25.401194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:25.402291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:25.402799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:25.403305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:25.404750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] RBAC: Adding query filter params (role.id=d5653c8adc4e4a88a006171b36c6db7e, user.id=1e6a18fb2f9142b19f7e4a7cb3c710c3, scope.system=all) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:25.405293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:25.407120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97f2063-c5d7-4b27-b8c4-643151c5055f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:25.418301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:25.421908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 172/334] 10.222.0.70 () {56 vars in 1308 bytes} [Mon May 23 13:09:25 2022] GET /identity/v3/role_assignments?user.id=1e6a18fb2f9142b19f7e4a7cb3c710c3&scope.system=all&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 510 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:27.195701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b34f3-dfa7-4669-a10f-4e9f2cbdeb58 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.196213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b34f3-dfa7-4669-a10f-4e9f2cbdeb58 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.196285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593b34f3-dfa7-4669-a10f-4e9f2cbdeb58 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.198083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 163/335] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:27.203321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd9304-0826-41fd-9c9b-c38800e8d89a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.204228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd9304-0826-41fd-9c9b-c38800e8d89a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.204742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd9304-0826-41fd-9c9b-c38800e8d89a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.233666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8cdd9304-0826-41fd-9c9b-c38800e8d89a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:27.235245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8cdd9304-0826-41fd-9c9b-c38800e8d89a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:27.329369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea28e-bfba-4de6-a943-241f9f3298c2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.329692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea28e-bfba-4de6-a943-241f9f3298c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.329816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 173/336] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:27.330244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7baea28e-bfba-4de6-a943-241f9f3298c2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.334933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 164/337] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:27.340135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.340431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.340756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.368692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea28e-bfba-4de6-a943-241f9f3298c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:27.371172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:27.375769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:27.535273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 174/338] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:27.562936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:27.580649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:27.595009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7baea28e-bfba-4de6-a943-241f9f3298c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:27.653384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:27.654571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.654856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.655135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.685679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:27.694810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:27.695730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:27.696514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:27.700981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 165/339] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:27.701979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:27.756946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:27.867472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c010222-9330-4aec-840a-6be6321af82f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:27.878786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0158337-600e-4f3b-9938-3cabf76d1a50 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:27.917346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:27.918046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.918552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.918846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.919620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:27.919936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:27.921379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:27.931134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 166/340] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:27 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:27.940215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:27.944594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:27.962396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:27.963283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:27.963570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:27.963848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:27.994428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:28.012444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:28.013421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:28.014617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 175/341] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 434 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:28.020804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:28.180441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76f9c04-ebe1-4a22-a4a4-03b10bdfd660 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:28.198844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ac8cb-789c-46d5-a438-d6776a91c8f1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:28.213650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:28.237756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:28.238840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:28.239573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:28.240002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:28.240941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:28.241396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:28.243072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:28.258665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 176/342] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:28.264632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:28.266730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:28.267403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:28.267691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:28.267975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:28.371431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:28.381498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:28.383930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:28.415522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65162d8b-7ead-4523-94ac-721debdec589 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:28.474516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e7bfbe-41b8-49de-95ca-167c2ea43565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:28.475469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7bfbe-41b8-49de-95ca-167c2ea43565 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:28.475917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7bfbe-41b8-49de-95ca-167c2ea43565 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:28.476339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e7bfbe-41b8-49de-95ca-167c2ea43565 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:28.540420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 167/343] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 600 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:28.557165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:28.561625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:28.581437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7bfbe-41b8-49de-95ca-167c2ea43565 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:28.583001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e7bfbe-41b8-49de-95ca-167c2ea43565 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:28.708966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7f5c3-4e1a-45ff-8d58-12a19db14c11 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:28.709185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 177/344] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:28.710298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221dcf1-7435-4298-b5b7-fec6396fb70c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:28.710755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221dcf1-7435-4298-b5b7-fec6396fb70c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:28.711308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:28.712292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221dcf1-7435-4298-b5b7-fec6396fb70c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:28.714597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 178/345] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:28 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:28.715493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221dcf1-7435-4298-b5b7-fec6396fb70c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:28.793620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-511ff6c7-2312-4afd-a04f-628be1b72629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:28.794811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ff6c7-2312-4afd-a04f-628be1b72629 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:28.795239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ff6c7-2312-4afd-a04f-628be1b72629 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:28.795657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-511ff6c7-2312-4afd-a04f-628be1b72629 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:28.798723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ff6c7-2312-4afd-a04f-628be1b72629 None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:28.800357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-511ff6c7-2312-4afd-a04f-628be1b72629 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:28.803697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 168/346] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:28 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:28.811317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:28.811818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:28.813248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:28.844809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:28.846548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:28.899095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:28.950465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221dcf1-7435-4298-b5b7-fec6396fb70c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:28.977932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 169/347] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:28.978821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.000193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea5213-8250-47ac-a576-72633988118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.001175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea5213-8250-47ac-a576-72633988118d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.001465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea5213-8250-47ac-a576-72633988118d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.001840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abea5213-8250-47ac-a576-72633988118d None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.003181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea5213-8250-47ac-a576-72633988118d None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:29.005397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abea5213-8250-47ac-a576-72633988118d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:29.008071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 179/348] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:28 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:29.015139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea5213-8250-47ac-a576-72633988118d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.159618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abea5213-8250-47ac-a576-72633988118d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:29.211003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:29.215136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e88364-af76-49ac-90c9-f5d412a807e2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:29.237108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.238173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.238466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.238746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.249145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:29.251140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:29.253922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 180/349] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:29 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:29.255714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.277867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0992b34a-ea94-4a4f-b44c-de258acdb1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.279170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b34a-ea94-4a4f-b44c-de258acdb1cf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.279550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b34a-ea94-4a4f-b44c-de258acdb1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.280505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0992b34a-ea94-4a4f-b44c-de258acdb1cf None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.281811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992b34a-ea94-4a4f-b44c-de258acdb1cf None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:29.284388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0992b34a-ea94-4a4f-b44c-de258acdb1cf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:29.287923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 170/350] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:28 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:29.292683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff586ca-f687-4909-930c-4c1f4858ab1b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.292683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff586ca-f687-4909-930c-4c1f4858ab1b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.292683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff586ca-f687-4909-930c-4c1f4858ab1b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.292683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 171/351] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:29.292683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff586ca-f687-4909-930c-4c1f4858ab1b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.295192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:29.425703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193f3810-5076-4545-90c9-34ae7e110842 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:29.448376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff586ca-f687-4909-930c-4c1f4858ab1b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:29.471842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.472692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.473150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.473741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.488443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:29.494699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.495378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.495666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.495944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:29.496482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:29.497369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:29.498558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:29.499186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=66be157488cf4d29902de7af0526cf6e) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:29.499186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:29.499321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 181/352] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:29.499387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cb7a05-a6d1-4b5e-a882-643ccbb6a3d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:29.499451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.507542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 172/353] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:29 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=a7511c9c66e546dda3a737b069282dae => generated 260 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:29.509182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef3ba9-9930-48ea-ad56-838e725f0646 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.510049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef3ba9-9930-48ea-ad56-838e725f0646 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.510351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adef3ba9-9930-48ea-ad56-838e725f0646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.534865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-adef3ba9-9930-48ea-ad56-838e725f0646 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:29.536760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-adef3ba9-9930-48ea-ad56-838e725f0646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:29.578827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:29.600737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:29.637343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebe7e86-b7cc-4401-a747-808b8e5812c8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:29.648600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 173/354] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:29.648600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6b01-7fc9-4953-beb7-bd2f3a195fc7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.648600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6b01-7fc9-4953-beb7-bd2f3a195fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.648600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1e6b01-7fc9-4953-beb7-bd2f3a195fc7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.649773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 174/355] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:29.650494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e6b01-7fc9-4953-beb7-bd2f3a195fc7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.687126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.687716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.687995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.688274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.689165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=66be157488cf4d29902de7af0526cf6e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:29.689586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:29.691109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b73f6a4-3d67-488c-8659-e0e35dc95c34 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:29.704718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 182/356] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:29 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 260 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:29.704718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cfe5f-b174-4643-821f-e27b176b46ff None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.704718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cfe5f-b174-4643-821f-e27b176b46ff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.704718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555cfe5f-b174-4643-821f-e27b176b46ff None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.704718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 183/357] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:29.707443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.708418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.709045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.732909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:29.734232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:29.804133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1e6b01-7fc9-4953-beb7-bd2f3a195fc7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:29.829400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 184/358] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:29.830298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:29.873115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:29.873803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.874084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.874355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.875141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:29.875446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:29.876921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2513e1bd-4547-4766-a6fc-e7a4c4cc1c61 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:29.888522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 175/359] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:29.889970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4243898-33a7-41db-abff-3244c407879a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:29.890260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4243898-33a7-41db-abff-3244c407879a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:29.890536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4243898-33a7-41db-abff-3244c407879a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:29.912690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4243898-33a7-41db-abff-3244c407879a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:29.914031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4243898-33a7-41db-abff-3244c407879a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:29.928403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:29.935302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:29.980150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b63ab-0813-4b3b-907b-569f9a2d2e9e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.012787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 176/360] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:30.014107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4243898-33a7-41db-abff-3244c407879a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.023299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.024109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.024392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.024694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.037788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:30.047484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:30.048479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 185/361] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:29 2022] GET /identity/v3/roles/reader => generated 84 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:30.049629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.151908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4243898-33a7-41db-abff-3244c407879a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.196298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200349d2-cf7a-49b8-a4ca-fe6d536b17e0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.222559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.224661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.225176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.225653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.237095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:30.242371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:30.246354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.247117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.247497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.248397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:30.252967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:30.259790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 177/362] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:30.260953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.281272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:30.356867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:30.359034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:30.399456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da4ba88-b802-4ec8-83f8-671151c4febd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.441528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.442311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.442772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.443061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.443829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:30.444133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:30.445555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:30.454712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 178/363] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:30.458929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.489199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 186/364] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:30.490216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.525186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:30.560680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:30.593354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543e17d-cae9-4275-a4ad-2f0417e2a147 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.619127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6090768d-1ea0-4d99-b6b4-1741ae564e75 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.642511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.643934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.644226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.644503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.660062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:30.662609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.663284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.663565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.663835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.665495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:30.665826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:30.667201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:30.668301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:30.670346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 179/365] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:30.672220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.681480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 187/366] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:30.682379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.797080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:30.810701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939fd7b4-cc5f-40a4-88af-6db0a918c173 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.811230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e373ae-7232-4ef2-9c51-50fc0f3c645c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:30.851552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.852443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.852750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.853026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] PATH_INFO: `/v3/users/a67591059b1c46a597f1eee7e20862fa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.854258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] RBAC: Authorizing `identity:get_user(user_id=a67591059b1c46a597f1eee7e20862fa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:30.856125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:30.857926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 180/367] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/users/a67591059b1c46a597f1eee7e20862fa => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:30.859175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.859853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:30.860950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:30.861379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:30.861999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:30.863308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:30.863516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:30.865868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:30.867760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 188/368] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:30.872778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:30.999253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2205cc5-a826-41ac-99a3-9b5874d2be92 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.004445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e9b35-34e4-4581-a1ef-a444fbc81ffd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.047961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.048558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.048884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.049135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.050759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.052437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.053258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.053682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.055547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:31.056012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.057577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.069109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 181/369] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:31.070197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.088624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:31.140911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.142567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.161629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:31.208287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a14db13-1866-462c-a1fd-00aea01a1fea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.250505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.251323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.251702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.252177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.253446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] RBAC: Adding query filter params (role.id=30181561a7e54d82b14a5ea28bafe082, user.id=a67591059b1c46a597f1eee7e20862fa, scope.system=all) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:31.253646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.255438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.276712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 182/370] 10.222.0.70 () {56 vars in 1308 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/role_assignments?user.id=a67591059b1c46a597f1eee7e20862fa&scope.system=all&role.id=30181561a7e54d82b14a5ea28bafe082 => generated 510 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:31.276942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.283310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 189/371] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:31.284840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.367362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:31.420515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a5d41dd-8c7a-4e35-a101-c35a39d1cee3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.426259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e46e68-fb68-44b9-92ee-6de089206be5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.473037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:31.483515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.484521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.484998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.485434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.495205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.496107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.496395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.506067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.508126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.512651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 183/372] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:31.515695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.621190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.623848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.686833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2429cfd-af90-44e3-8c3f-7ad190329d95 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.705802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:31.737670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.738562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.738886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.739695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.741200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.743434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.746234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 184/373] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:31.751309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.776718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 190/374] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 487 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:31.778202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.850468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:31.904965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5602ee6c-c10b-4199-a1c8-2fece508dc07 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.917852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e634aa6c-da46-43df-ae67-dd845a987e70 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:31.954913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.955926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.956412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.957084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.959501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:31.959818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.961301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.967541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 185/375] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 260 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:31.968487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:31.971165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:31.972610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:31.973104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:31.974594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:31.976233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:31.977951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:31.981091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 191/376] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:31.991003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:32.062497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:32.204691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4128ea4-0369-4fd8-b1cf-152e825a60b7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:32.214969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f62cd5-85fb-4fc3-bb0c-bafdf46bfecd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:32.233580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:32.248846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b813a141-3f05-47b8-baea-417b053ae775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:32.251774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813a141-3f05-47b8-baea-417b053ae775 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.251774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813a141-3f05-47b8-baea-417b053ae775 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.251774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b813a141-3f05-47b8-baea-417b053ae775 None admin] PATH_INFO: `/v3/projects/8db400662d744ef19141527de1d2418c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.252407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813a141-3f05-47b8-baea-417b053ae775 None admin] RBAC: Authorizing `identity:get_project(project_id=8db400662d744ef19141527de1d2418c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:32.253627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b813a141-3f05-47b8-baea-417b053ae775 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:32.257687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 186/377] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/projects/8db400662d744ef19141527de1d2418c => generated 344 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:32.257928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe62c67-fe53-496d-855b-cb79900748f2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.258177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe62c67-fe53-496d-855b-cb79900748f2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.258460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe62c67-fe53-496d-855b-cb79900748f2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.262188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 187/378] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:32.263455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe62c67-fe53-496d-855b-cb79900748f2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:32.274116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:32.275636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.275946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.276231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.277125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:32.277442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:32.279600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0987a461-c35b-4d92-b147-cb25a4f296ab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:32.287678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 192/379] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:31 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=a7511c9c66e546dda3a737b069282dae => generated 260 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:32.289085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10b00-c0d4-42a8-8688-85b078074912 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.289377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10b00-c0d4-42a8-8688-85b078074912 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.289654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fe10b00-c0d4-42a8-8688-85b078074912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.309106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4fe10b00-c0d4-42a8-8688-85b078074912 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:32.310759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4fe10b00-c0d4-42a8-8688-85b078074912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:32.402165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 193/380] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:32.403377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc2ed-d00e-4d6e-922f-4e330f9a970b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.403927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc2ed-d00e-4d6e-922f-4e330f9a970b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.404210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6cc2ed-d00e-4d6e-922f-4e330f9a970b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.413173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 194/381] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:32 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:32.419470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.419981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.420410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.444883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe62c67-fe53-496d-855b-cb79900748f2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:32.456551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:32.458314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:32.477884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:32.493227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:32.494577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.496720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.496720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] PATH_INFO: `/v3/groups/3f8ab35e23144d588f101e5bdf5caae2` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.503333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] RBAC: Authorizing `identity:get_group(group_id=3f8ab35e23144d588f101e5bdf5caae2)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:32.506254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:32.511747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 188/382] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:32 2022] GET /identity/v3/groups/3f8ab35e23144d588f101e5bdf5caae2 => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:32.512822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:32.546048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 195/383] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:32.547234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:32.664989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:32.698695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a91d41f-e332-4581-8626-39a4a250144d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:32.720012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590bdb3c-365a-4ce2-aec3-a7855e5cdcbd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:32.759699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:32.760800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.761233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.761655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.774048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:32.778592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:32.779817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:32.780331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:32.780857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:32.781903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] RBAC: Adding query filter params (group.id=3f8ab35e23144d588f101e5bdf5caae2, role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=8db400662d744ef19141527de1d2418c) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:32.782409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:32.783504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:32.784390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:32.785192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 189/384] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:32 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:32.786397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:32.787921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:32.801217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 196/385] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:32 2022] GET /identity/v3/role_assignments?group.id=3f8ab35e23144d588f101e5bdf5caae2&scope.project.id=8db400662d744ef19141527de1d2418c&role.id=33de09388f974cd391e13bb4f2502a52 => generated 260 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:32.801217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:32.842067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.026412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc898d11-efe6-4f82-8060-080eabfdab7a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.034263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fee7196-f834-4f9b-9c11-b83791814eaa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.055322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.090031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.091976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.093480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.094380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.095350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:33.096711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.097272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.098308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.099972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:33.100492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.100989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.115009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 197/386] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:33.118403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:33.120611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:33.126522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:33.127451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:33.128714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 190/387] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:33.134373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:33.232944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.272521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0388ba91-2d58-4475-baff-1ad1ced9137f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.333308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.335066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.335494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.335925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.338250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.339988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:33.342192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 198/388] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:33 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:33.349876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:33.355302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b31a7-bc69-4dcd-bd5e-1609b67db0e2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.459606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65183b18-5dc7-437b-bc9d-711006c26629 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.466031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.466740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.467026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.467307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.468095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:33.468408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.469927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:33.488317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 191/389] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 354 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:33.497625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:33.512555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.513525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.513815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.514090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.515525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.517009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:33.518720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 199/390] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:33 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:33.526455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:33.557580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.585050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.714112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc22205-7c1d-4e37-9300-51a3360995a1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.720035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0242184d-8d76-48c6-acc7-d0d73f9563cd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.762821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.763720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.764002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.764275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.766626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=66be157488cf4d29902de7af0526cf6e, group_id=9ecca04901ac4c4a96ce7af4039dc8be, role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.768087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:33.779104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.780111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.786284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.786284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.786284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.786284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:33.786877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 192/391] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:33 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:33.791465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:33.852884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 200/392] 10.222.0.70 () {58 vars in 1442 bytes} [Mon May 23 13:09:33 2022] PUT /identity/v3/projects/66be157488cf4d29902de7af0526cf6e/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/a7511c9c66e546dda3a737b069282dae => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:33.938962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c687e374-3f4d-4605-8d30-3192e1559265 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:33.950071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:33.990928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02041899-34df-4855-aee6-b2aa23f1fdce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:33.991839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02041899-34df-4855-aee6-b2aa23f1fdce None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:33.992123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02041899-34df-4855-aee6-b2aa23f1fdce None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:33.992455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02041899-34df-4855-aee6-b2aa23f1fdce None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:33.997582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02041899-34df-4855-aee6-b2aa23f1fdce None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:33.999344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02041899-34df-4855-aee6-b2aa23f1fdce None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:34.003186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 193/393] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:33 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:34.010612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:34.120336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:34.155992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e1a69c-b3f3-4509-baff-1da52b24e3f7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:34.193302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6248ccb-857f-42c3-abcd-f37288e53228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:34.194141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6248ccb-857f-42c3-abcd-f37288e53228 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:34.194425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6248ccb-857f-42c3-abcd-f37288e53228 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:34.194700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6248ccb-857f-42c3-abcd-f37288e53228 None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:34.200592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6248ccb-857f-42c3-abcd-f37288e53228 None admin] RBAC: Authorizing `identity:create_grant(project_id=66be157488cf4d29902de7af0526cf6e, group_id=9ecca04901ac4c4a96ce7af4039dc8be, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:34.200592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6248ccb-857f-42c3-abcd-f37288e53228 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:34.286482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 201/394] 10.222.0.70 () {58 vars in 1442 bytes} [Mon May 23 13:09:34 2022] PUT /identity/v3/projects/66be157488cf4d29902de7af0526cf6e/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:34.937381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff24513-b1aa-44d0-8f46-3bd299fad55a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:34.938436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff24513-b1aa-44d0-8f46-3bd299fad55a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:34.938926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff24513-b1aa-44d0-8f46-3bd299fad55a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:34.940799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 194/395] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:34.947106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:34.947997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:34.948498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:34.967450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:34.969068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:35.007488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973a898-e48e-4176-aae6-a26b683e1348 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.008139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973a898-e48e-4176-aae6-a26b683e1348 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.008649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9973a898-e48e-4176-aae6-a26b683e1348 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.011888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 195/396] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:35.016725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.017288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.017754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.045532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 202/397] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:35.053648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:35.055110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:35.069630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.134860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 196/398] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:35.144590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.174344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.180168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5cf5ff-4b6c-48e7-a6f0-6a7007861b2e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.216482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.217433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.217802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.218089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.230113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:35.239290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:35.240816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:35.241624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 203/399] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:35.246901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.373181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d091d9-faf3-40b2-85da-66cb02615700 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.388677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b894abf-b824-4fcd-8852-c59c9d91cad7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.397656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.413377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.414302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.414757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.415177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.416088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:35.416542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:35.418286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.418836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:35.428971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 204/400] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:35.434723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.461415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.462143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.462428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.462701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.485296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:35.493345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:35.494449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:35.495526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 197/401] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:35.500413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.606653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda95897-488b-4501-853c-b0862b7a2898 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.631432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-467c02a6-cc3d-4fbe-b1b5-b8c9c3633934 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.653803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.655769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.656293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.658335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.660154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:35.665582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:35.671156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 205/402] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:35.673375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.712482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.716328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.716886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.717356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.718362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:35.720052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:35.721740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:35.739787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 198/403] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:35.744169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.774291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.790550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:35.836318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074d6bb7-a431-4564-943e-8004df388695 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.872081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.873061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.873271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.873569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.874747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:35.875885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0678e39-d96e-4611-bb28-6101411b92fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:35.876210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:35.878253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 206/404] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:35.883956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:35.914469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6de976-4925-417e-a2aa-54cfd27a3b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:35.915919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6de976-4925-417e-a2aa-54cfd27a3b88 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.916477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6de976-4925-417e-a2aa-54cfd27a3b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.917040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6de976-4925-417e-a2aa-54cfd27a3b88 None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.918535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6de976-4925-417e-a2aa-54cfd27a3b88 None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:35.920344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6de976-4925-417e-a2aa-54cfd27a3b88 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:35.922750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 199/405] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:35.924350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb9269-35a7-4f1e-804b-cc1bb8c74c89 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:35.924891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb9269-35a7-4f1e-804b-cc1bb8c74c89 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:35.925491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9cb9269-35a7-4f1e-804b-cc1bb8c74c89 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:35.927575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 200/406] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:35.930159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb9269-35a7-4f1e-804b-cc1bb8c74c89 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.013147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775dea-b9bf-4cd8-b515-2ea3f11e5d43 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:36.042706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:36.048098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:36.052084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9cb9269-35a7-4f1e-804b-cc1bb8c74c89 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:36.068718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:36.070891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.071395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.071898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.074795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0c0feadab4f6894f27b847cbb48d1, group_id=9ecca04901ac4c4a96ce7af4039dc8be, role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:36.076941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:36.091889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f49f40-5b6b-4599-9801-b5f65f0a28d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:36.092793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49f40-5b6b-4599-9801-b5f65f0a28d1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.093082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49f40-5b6b-4599-9801-b5f65f0a28d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.093362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f49f40-5b6b-4599-9801-b5f65f0a28d1 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.094637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49f40-5b6b-4599-9801-b5f65f0a28d1 None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:36.096084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f49f40-5b6b-4599-9801-b5f65f0a28d1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:36.098032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 201/407] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:35 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:36.104267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.104526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.104832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.123709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:36.124989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:36.200331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 207/408] 10.222.0.70 () {58 vars in 1442 bytes} [Mon May 23 13:09:35 2022] PUT /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/a7511c9c66e546dda3a737b069282dae => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:09:36.202334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.213025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 202/409] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:36.234728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.339698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:36.349530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3995c82-2d2e-4fb8-bdd7-5ff9e6b646fa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:36.363007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af356c45-24e8-43cf-a42a-52b1af97b4ab None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:36.389011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:36.389827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.390109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.390383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.402690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:36.410953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:36.411917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:36.412828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 203/410] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:36.416844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:36.418312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.418716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.419195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.419685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.425487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0c0feadab4f6894f27b847cbb48d1, group_id=9ecca04901ac4c4a96ce7af4039dc8be, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:36.427885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:36.557240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 208/411] 10.222.0.70 () {58 vars in 1442 bytes} [Mon May 23 13:09:36 2022] PUT /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1/groups/9ecca04901ac4c4a96ce7af4039dc8be/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:36.557972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2be60c-881e-47fb-ba2b-085c14652ae9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:36.599420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:36.600039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.600327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.600637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.601414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:36.601727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:36.603141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862681a4-658f-4f42-be40-f408cf0e4894 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:36.612944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 204/412] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:36.619279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.701320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:36.730604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b967dfe-6b02-4224-8e53-3f3ed1f12e87 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.731227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b967dfe-6b02-4224-8e53-3f3ed1f12e87 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.731697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b967dfe-6b02-4224-8e53-3f3ed1f12e87 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.733647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 205/413] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:36.740540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.741205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.741699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.770396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:36.772815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:36.789511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:36.821370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f81c66e-54fe-4431-a7df-7c8a62005e14 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:36.865584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:36.866474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:36.866750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:36.867030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] PATH_INFO: `/v3/groups/3f8ab35e23144d588f101e5bdf5caae2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:36.868992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] RBAC: Authorizing `identity:get_group(group_id=3f8ab35e23144d588f101e5bdf5caae2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:36.869404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 206/414] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:36.870452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:36.872213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 209/415] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:36 2022] GET /identity/v3/groups/3f8ab35e23144d588f101e5bdf5caae2 => generated 229 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:36.877621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.911320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:36.984678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea5378e-4103-4d5b-b0f6-28acb996ef47 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:37.022626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:37.023139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.023424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.023703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] PATH_INFO: `/v3/projects/8db400662d744ef19141527de1d2418c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.028095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] RBAC: Authorizing `identity:get_project(project_id=8db400662d744ef19141527de1d2418c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:37.028095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:37.028709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 207/416] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:36 2022] GET /identity/v3/projects/8db400662d744ef19141527de1d2418c => generated 344 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:37.034281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:37.058586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:37.116727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe74712-10ce-4ae1-87cb-83f7900aebd9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:37.137476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:37.141112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ea6b2-4269-484e-968c-4a19e0dd4ea4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:37.177902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b25dc3-1494-45f2-8b75-2cc8632736f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:37.179197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25dc3-1494-45f2-8b75-2cc8632736f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.179689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25dc3-1494-45f2-8b75-2cc8632736f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.180164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b25dc3-1494-45f2-8b75-2cc8632736f2 None admin] PATH_INFO: `/v3/projects/8db400662d744ef19141527de1d2418c/groups/3f8ab35e23144d588f101e5bdf5caae2/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.185962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25dc3-1494-45f2-8b75-2cc8632736f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=8db400662d744ef19141527de1d2418c, group_id=3f8ab35e23144d588f101e5bdf5caae2, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:37.185962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b25dc3-1494-45f2-8b75-2cc8632736f2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:37.211688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:37.212801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.213161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.213556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.227371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:37.236382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:37.237358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:37.238142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 210/417] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 328 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:37.245793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:37.292678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 208/418] 10.222.0.70 () {58 vars in 1442 bytes} [Mon May 23 13:09:37 2022] PUT /identity/v3/projects/8db400662d744ef19141527de1d2418c/groups/3f8ab35e23144d588f101e5bdf5caae2/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:37.413609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0185c33d-71b7-4bb3-ab25-e11830e783e2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:37.462376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81412a0d-46d0-4c9e-843d-b7cfa9b9bc97 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.462712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81412a0d-46d0-4c9e-843d-b7cfa9b9bc97 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.462950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81412a0d-46d0-4c9e-843d-b7cfa9b9bc97 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.464769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 209/419] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:37.471989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.472307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.472560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.487988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:37.489968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.490284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.490571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.491368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:37.491689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:37.493165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:37.513165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:37.514404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:37.514613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 211/420] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:37.520554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:37.577509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:37.652590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 210/421] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:37.678704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0194a7db-d877-419f-83cd-0c29b90eeb67 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:37.709837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:37.712138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:37.741431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:37.742159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.742447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.742728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.852816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d11c741-95b2-4136-9afd-02034c735203 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:37.893821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:37.895374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:37.898608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:37.899475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:37.899766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:37.900045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:37.914101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:37.928986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:37.929979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:37.930778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 211/422] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:37.936470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:37.963766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:38.017002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:38.068195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5eae668-6da9-49f0-afeb-005d3773c186 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:38.115557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:38.116336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:38.116732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:38.117081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:38.118562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:38.119100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:38.120665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:38.136377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 212/423] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:38.138300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:38.156228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 212/424] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 636 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:38.161763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:38.326119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d7e007-88c4-4cd6-9950-358d0c44c478 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:38.349933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a0366-54a1-4a80-9c76-dc1cbff7c94c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:38.371641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:38.391422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:38.392374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:38.392862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:38.393430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:38.404321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:38.415591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:38.416385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:38.416728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:38.417009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:38.418250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:38.419716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:38.421878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 213/425] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:38 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:38.428847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:38.527684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:38.529775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:38.634231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf21c04c-6412-4dca-a4b3-d61b12b25add None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:38.675464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 213/426] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 538 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:38.681651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:38.689915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:38.691207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:38.691783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:38.692256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:38.693817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:38.695610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:38.697787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 214/427] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:38 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:38.703840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:38.726138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:38.815593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:38.820603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f847a546-0efc-4309-917d-e5d219da2735 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:38.856641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:38.857886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:38.858180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:38.858467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] PATH_INFO: `/v3/projects/66be157488cf4d29902de7af0526cf6e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:38.859707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] RBAC: Authorizing `identity:get_project(project_id=66be157488cf4d29902de7af0526cf6e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:38.861375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:38.868160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 214/428] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:38 2022] GET /identity/v3/projects/66be157488cf4d29902de7af0526cf6e => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:38.869424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:38.925394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e4461f-adb1-4005-9cf4-905a021161ac None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:38.997541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:38.998278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:38.998571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:38.998854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:38.999724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=66be157488cf4d29902de7af0526cf6e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:39.000044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:39.001562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:39.005668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:39.016965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c20979-4542-4872-a4eb-2bae35794bda None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.036105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 215/429] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:38 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=a7511c9c66e546dda3a737b069282dae => generated 611 bytes in 332 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:39.060337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da33f0fd-67de-4f9c-afc1-709dc790e8ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:39.061357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33f0fd-67de-4f9c-afc1-709dc790e8ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.061648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33f0fd-67de-4f9c-afc1-709dc790e8ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.061929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da33f0fd-67de-4f9c-afc1-709dc790e8ca None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.063731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33f0fd-67de-4f9c-afc1-709dc790e8ca None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:39.065318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da33f0fd-67de-4f9c-afc1-709dc790e8ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:39.071674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 215/430] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:38 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:39.074511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:39.219180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c26ff01-8b5e-4a19-9958-2b590d915885 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.220039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8212-29ba-4a78-84e3-c17d4a8f4b85 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.220556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8212-29ba-4a78-84e3-c17d4a8f4b85 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.221067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590f8212-29ba-4a78-84e3-c17d4a8f4b85 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.223579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 216/431] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:39.230060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f1cf-6a58-452f-a1b3-3b8399bd46d7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.230495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f1cf-6a58-452f-a1b3-3b8399bd46d7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.230870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b305f1cf-6a58-452f-a1b3-3b8399bd46d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.249117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b305f1cf-6a58-452f-a1b3-3b8399bd46d7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:39.250715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b305f1cf-6a58-452f-a1b3-3b8399bd46d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:39.265024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:39.279861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:39.285606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.285606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.285606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.285606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=66be157488cf4d29902de7af0526cf6e) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:39.285606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:39.285606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:39.294451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 216/432] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:39 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=66be157488cf4d29902de7af0526cf6e&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 611 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:39.343462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 217/433] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:39.371121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:39.454109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb37d1d-c633-45d3-bc02-8c9f2f3a68d7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.454451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb37d1d-c633-45d3-bc02-8c9f2f3a68d7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.454669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfb37d1d-c633-45d3-bc02-8c9f2f3a68d7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.456229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 218/434] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:39.463515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.464152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.464668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.486442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:39.488759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:39.536116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3dbbe2f-de47-4717-b2d6-7ede98ba48c4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.578805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:39.579612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.579894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.580170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.591942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 219/435] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:39.596691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:39.601895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:39.602773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:39.603567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 217/436] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:39.610305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:39.620068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:39.636005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:39.692013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:39.762253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da50dd1-d7ad-45e5-9147-6d3bdfb4a53a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.767292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68ff4a5-cff5-4002-9bf2-f8b667b5bba3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.803342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:39.804125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.804416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.804760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.805520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] RBAC: Adding query filter params (name=member) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:39.805833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:39.807281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:39.810295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:39.811123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.811403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.811674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:39.817049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 220/437] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:09:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:39.822694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:39.825212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:39.833909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:39.834896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:39.835691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 218/438] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:39.840868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:39.921603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:39.944478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-419e4180-97e7-4805-a4be-e9701b1fde80 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.964418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3150da-f8ef-4208-8684-579f15b124fc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:39.982173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:39.992084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6610-d535-416d-8b53-fcf730094e13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:39.992784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6610-d535-416d-8b53-fcf730094e13 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:39.993072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6610-d535-416d-8b53-fcf730094e13 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:39.993348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6610-d535-416d-8b53-fcf730094e13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.004336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.005121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.005405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.005685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.006428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:40.006741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.008138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2669fc21-4972-4e0e-91c3-50090b267de4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.016844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 219/439] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:40.017924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21d3fc-4c3e-430b-b711-3135700ba33e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.018211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21d3fc-4c3e-430b-b711-3135700ba33e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.018482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc21d3fc-4c3e-430b-b711-3135700ba33e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.019840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 220/440] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:40.022166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21d3fc-4c3e-430b-b711-3135700ba33e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.067814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b6610-d535-416d-8b53-fcf730094e13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.069310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b6610-d535-416d-8b53-fcf730094e13 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.127082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc21d3fc-4c3e-430b-b711-3135700ba33e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.150034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.166752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.167412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.167700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.167981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.169084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 221/441] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:40.170358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.170656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.170928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.187391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:40.188629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:40.224672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.247318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.248812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.255061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 222/442] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:40.257078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.351409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 221/443] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 23 13:09:40.352262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.360086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5b3e9e-aaec-42e7-b021-d689effa1566 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.373260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.393206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.394066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.398536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.398536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.398536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.398536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.399507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 223/444] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:40.400260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.432795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.456530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0a5687-a334-4c33-827c-c97a18ad2256 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.493485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.494280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.494562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.494836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.510311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:40.517258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa9f6be-58f1-44cf-98fe-173da1e886ac None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:40.520971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:40.521864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:40.522651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 222/445] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:40.523317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.553791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98450866-f106-4544-9457-d96db3952ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.554669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98450866-f106-4544-9457-d96db3952ceb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.554944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98450866-f106-4544-9457-d96db3952ceb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.555218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98450866-f106-4544-9457-d96db3952ceb None admin] PATH_INFO: `/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.556378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98450866-f106-4544-9457-d96db3952ceb None admin] RBAC: Authorizing `identity:get_project(project_id=d7c0c0feadab4f6894f27b847cbb48d1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.557857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98450866-f106-4544-9457-d96db3952ceb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.559536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 224/446] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/projects/d7c0c0feadab4f6894f27b847cbb48d1 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:40.560283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98450866-f106-4544-9457-d96db3952ceb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.610345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.629814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214ec156-9e0d-4d0e-a579-e9c6f95c35e3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.662295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98450866-f106-4544-9457-d96db3952ceb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.665095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.665955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.666241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.666516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.667708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.669273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.670892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 223/447] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:40.671665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.675517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.698944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.699540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.699821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.700101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.700942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:40.701298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.702615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.711134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 225/448] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:40.711927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.777712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcea83c7-7d21-4b2b-b45c-ddafa353b2be None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.809977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.810834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.811111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.811381 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] PATH_INFO: `/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.812374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549403f6-9f37-42a8-a006-10ed233d2a37 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.812552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] RBAC: Authorizing `identity:get_group(group_id=9ecca04901ac4c4a96ce7af4039dc8be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.814007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.815623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 224/449] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/groups/9ecca04901ac4c4a96ce7af4039dc8be => generated 226 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:40.817035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.824306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.853948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.854614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.854904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.855186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:40.856024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=d5653c8adc4e4a88a006171b36c6db7e, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:40.856347 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:40.857828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:40.864766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 226/450] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=d5653c8adc4e4a88a006171b36c6db7e => generated 611 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:40.865608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:40.914128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:40.948359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7ceb7c-c32f-4e0a-a6f6-c8345eba582b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.985967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89fdc856-2d44-415d-b2de-455085e980de None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:40.987888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:40.989175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:40.989637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:40.989927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:41.027972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:41.029189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:41.029716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:41.029716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:41.030613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] RBAC: Adding query filter params (group.id=9ecca04901ac4c4a96ce7af4039dc8be, role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:41.030875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:41.032369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:41.048395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 227/451] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/role_assignments?group.id=9ecca04901ac4c4a96ce7af4039dc8be&scope.project.id=d7c0c0feadab4f6894f27b847cbb48d1&role.id=a7511c9c66e546dda3a737b069282dae => generated 611 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:41.074686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:41.076187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:41.166781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:41.208675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 225/452] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:41.214767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:41.264156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:41.329134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67114af3-55f2-4678-a02f-c55ad6a66ca0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:41.367284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:41.368164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:41.368446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:41.368747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] PATH_INFO: `/v3/projects/8db400662d744ef19141527de1d2418c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:41.370078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] RBAC: Authorizing `identity:get_project(project_id=8db400662d744ef19141527de1d2418c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:41.371386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:41.373135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 228/453] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:09:41 2022] GET /identity/v3/projects/8db400662d744ef19141527de1d2418c => generated 344 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:41.379707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:41.485288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb2929c-1c85-4e19-bff4-90f70c46eb08 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:41.518786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bda6ab-7ca1-43fd-ad94-268345190212 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:41.519577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda6ab-7ca1-43fd-ad94-268345190212 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:41.519866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda6ab-7ca1-43fd-ad94-268345190212 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:41.520146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bda6ab-7ca1-43fd-ad94-268345190212 None admin] PATH_INFO: `/v3/groups/3f8ab35e23144d588f101e5bdf5caae2` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:41.521343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bda6ab-7ca1-43fd-ad94-268345190212 None admin] RBAC: Authorizing `identity:get_group(group_id=3f8ab35e23144d588f101e5bdf5caae2)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:41.522785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47bda6ab-7ca1-43fd-ad94-268345190212 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:41.524402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 226/454] 10.222.0.70 () {56 vars in 1180 bytes} [Mon May 23 13:09:41 2022] GET /identity/v3/groups/3f8ab35e23144d588f101e5bdf5caae2 => generated 229 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:41.530637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:41.626791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4127a4a7-9f33-4e85-992c-2ec7c20c6ad7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:41.633607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:41.659744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:41.661034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:41.661315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:41.661589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:41.662409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] RBAC: Adding query filter params (group.id=3f8ab35e23144d588f101e5bdf5caae2, role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=8db400662d744ef19141527de1d2418c) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:41.662715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:41.664141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19cbb657-f7c6-4377-82f3-72ba6214b615 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:41.669819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 229/455] 10.222.0.70 () {56 vars in 1376 bytes} [Mon May 23 13:09:41 2022] GET /identity/v3/role_assignments?group.id=3f8ab35e23144d588f101e5bdf5caae2&scope.project.id=8db400662d744ef19141527de1d2418c&role.id=33de09388f974cd391e13bb4f2502a52 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:44.269783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be970c-66b8-49ba-8f17-f4944dca2dd8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.270797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be970c-66b8-49ba-8f17-f4944dca2dd8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.271287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59be970c-66b8-49ba-8f17-f4944dca2dd8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.272940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 227/456] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:44.286784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.287298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.287377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.312918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:44.314148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:44.413511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 230/457] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:44.459738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be970c-66b8-49ba-8f17-f4944dca2dd8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:44.542456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:44.618318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59be970c-66b8-49ba-8f17-f4944dca2dd8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:44.674608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:44.675485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.675847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.676393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.689983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:44.691849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e38e834f-7cec-4e78-bfd3-6e0eea6fad46 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:44.716955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:44.717911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:44.718815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:44.718815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:44.718815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:44.718815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:44.718815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 228/458] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:44.724455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:44.734807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74f32-26ec-4065-8be4-aec6cd9402df None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.735102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74f32-26ec-4065-8be4-aec6cd9402df None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.735383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e74f32-26ec-4065-8be4-aec6cd9402df None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.737031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 229/459] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:44.742756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb506b4-f748-4ab2-b605-ddce59d6138c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.743049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb506b4-f748-4ab2-b605-ddce59d6138c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.743325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb506b4-f748-4ab2-b605-ddce59d6138c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.760898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3cb506b4-f748-4ab2-b605-ddce59d6138c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:44.762102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3cb506b4-f748-4ab2-b605-ddce59d6138c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:44.866274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6754886-685a-4c62-8ff7-068898eca423 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:44.868909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:44.881640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 230/460] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:44.898054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ae5d-8015-4741-8f62-f288b509948d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.898341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ae5d-8015-4741-8f62-f288b509948d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.898624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab35ae5d-8015-4741-8f62-f288b509948d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.900763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 231/461] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:09:44.912994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.913331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.913648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.922100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:44.922841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:44.923128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:44.923406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:44.924160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:44.924467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:44.926120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:44.935898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:44.937542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:44.947224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 231/462] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:44.947911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:44.959348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.036718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 232/463] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:45.036718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:45.090696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded5f522-8b75-4536-af07-7665af74b96b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.139683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.140517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.140836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.141118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.153398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.154892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.162774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4236d2d-f98c-4d07-8b7a-9b8d624309e0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:45.175249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:45.176426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:45.177524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:45.177524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:45.177524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:45.177524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:45.179103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 232/464] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:45.180186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:45.202455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.207845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.208527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.208840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.209115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.209822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.211279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.290375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653be075-bb59-437d-81aa-b866bef114ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.324711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.325505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.325786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.326055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.336812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.338324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8bc09ae2-7114-4f2f-b346-e7f8f6d1f0d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:45.356477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:45.357495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:45.358315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:45.358315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:45.358315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:45.358315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:45.358315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 233/465] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:45.364388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f6930-6fd2-4ba2-a7fc-72bf1a2c259c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.364388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f6930-6fd2-4ba2-a7fc-72bf1a2c259c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.364388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-441f6930-6fd2-4ba2-a7fc-72bf1a2c259c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.364388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 234/466] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:45.364388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f6930-6fd2-4ba2-a7fc-72bf1a2c259c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:45.364879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 233/467] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:45 2022] POST /identity/v3/users => generated 246 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:45.367100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:45.380998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.457704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.493222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af15000-6bee-40f6-bfe5-90975d3045dc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.503164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-441f6930-6fd2-4ba2-a7fc-72bf1a2c259c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.539860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.541558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.542236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.542869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.544023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:45.544763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.546686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6b9d32-bcb0-4679-ae73-6258c62054d7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.547143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.547910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.548285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.548688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.549586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:45.549907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.552797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.572393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 234/468] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:45.572609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 235/469] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:45.572711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.572711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.572711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.579120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:45.598562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:45.600278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:45.686970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.696382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 235/470] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:45.704353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:45.718278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-356ffa6f-3841-4299-bb6c-33d5e2de4923 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.748163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:45.767621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.769499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.770180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.770934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.771874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.774256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.835141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28a6eff-99ef-425f-bc0e-49a884aea2cd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:45.877179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec130d0c-4dc2-44d4-aec7-4a1c425bd603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:45.878331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec130d0c-4dc2-44d4-aec7-4a1c425bd603 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:45.878846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec130d0c-4dc2-44d4-aec7-4a1c425bd603 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:45.879301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec130d0c-4dc2-44d4-aec7-4a1c425bd603 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:45.880190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec130d0c-4dc2-44d4-aec7-4a1c425bd603 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:45.881878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec130d0c-4dc2-44d4-aec7-4a1c425bd603 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:45.930839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 236/471] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:45 2022] POST /identity/v3/users => generated 248 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:45.931686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:46.049255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 236/472] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:45 2022] POST /identity/v3/users => generated 248 bytes in 352 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:46.057414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089af7f-6a29-4353-88f1-23edd3f56f84 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.057630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089af7f-6a29-4353-88f1-23edd3f56f84 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.057893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9089af7f-6a29-4353-88f1-23edd3f56f84 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.059696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 237/473] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:46 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:46.070134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.070423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.070702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.090537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550439a-e93b-475a-bd65-deaa5752d909 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:46.111920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:46.115492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:46.168253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:46.169592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.170917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.171865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.186339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:46.187981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:46.219259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:46.220162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:46.221050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:46.221050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:46.221050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:46.221050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:46.224054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 237/474] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 292 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:46.230446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:46.233825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:46.263218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 238/475] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:46.291232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:46.293761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:46.388973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca197df6-8c56-4097-a35b-406aa576e777 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:46.430825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:46.431515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.431794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.432260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.432849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:46.433163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:46.434566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:46.439238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f4a514-fb03-46b1-906c-769e4a38e323 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:46.449340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 238/476] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:46.455541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:46.483441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:46.484932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.485532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.486060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.486980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:46.489092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:46.537228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.sql.core [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 13:09:46.537228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 13:09:46.537228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [parameters: {'id': '4fa93e8bbab44d6a8ffabc1a8bd2ede5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 13:09:46.537228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80396) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result = self._query(query) May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application conn.query(q) May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result.read() May 23 13:09:46.560029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return ref.to_dict() May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 13:09:46.561161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application next(self.gen) May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application yield resource May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application next(self.gen) May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.session.rollback() May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.force_reraise() May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise self.value May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application session.commit() May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 13:09:46.562336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._prepare_impl() May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.session.flush() May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._flush(objects) May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application compat.raise_( May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application flush_context.execute() May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rec.execute(self) May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 13:09:46.563280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 13:09:46.564371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result = self._query(query) May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application conn.query(q) May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result.read() May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 13:09:46.569614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [parameters: {'id': '4fa93e8bbab44d6a8ffabc1a8bd2ede5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:46.570663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 23 13:09:46.571987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:46.579515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:46.579652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 239/477] 10.222.0.70 () {60 vars in 1131 bytes} [Mon May 23 13:09:46 2022] POST /identity/v3/roles => generated 144 bytes in 284 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:46.580499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcd4dff-4c1d-4254-aaed-93a1773f28c1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:46.582989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:46.618484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:46.619170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.619507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.619796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.620523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:46.623196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:46.676882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:46.739479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15bc176-edb1-4870-ae12-364f686c253a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:46.765410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 239/478] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:46 2022] POST /identity/v3/users => generated 249 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:46.779179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab267c3-f0ac-4005-8382-ab58390d24cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:46.780330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab267c3-f0ac-4005-8382-ab58390d24cb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:46.780815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab267c3-f0ac-4005-8382-ab58390d24cb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:46.781234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab267c3-f0ac-4005-8382-ab58390d24cb None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:46.797795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab267c3-f0ac-4005-8382-ab58390d24cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bab267c3-f0ac-4005-8382-ab58390d24cb None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:46.809268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:46.810453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 23 13:09:46.811443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:46.814001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 240/479] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:09:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:46.821421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:46.980141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b5b663-5c4d-4e1f-a70d-0ff1d55bf1b0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:47.063993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:47.064772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:47.065064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:47.065346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:47.066998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:47.067317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:47.068779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5730fbc8-189b-40c6-b023-1d695ee44cfe None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:47.078299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 240/480] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:48.094900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ef29-1c17-40b4-ac32-64b270c2ad84 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:48.095775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ef29-1c17-40b4-ac32-64b270c2ad84 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:48.096195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1442ef29-1c17-40b4-ac32-64b270c2ad84 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:48.098606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 241/481] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:48.103970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:48.104868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:48.105357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:48.123548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:48.125111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:48.129344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:48.217738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 241/482] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:48.263650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ef29-1c17-40b4-ac32-64b270c2ad84 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:48.398737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1442ef29-1c17-40b4-ac32-64b270c2ad84 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:48.409056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:48.437319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:48.438132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:48.438414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:48.438690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:48.450442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:48.459940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:48.461800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:48.462955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 242/483] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:48.470207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:48.630992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0aad65-9a48-4bb7-94a6-f21f2661d386 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:48.692015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:48.692649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:48.692938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:48.693216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:48.693979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:48.694287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:48.695714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:48.709318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 242/484] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:48.714859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:48.858484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3d8d9b-8e7a-45a7-822b-23a05d116b6b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:48.915720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:48.917232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:48.917773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:48.918272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:48.933431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:48.935824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d0e25a3b-6624-4faa-83db-fc0c8cdad9fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:48.969084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:48.969953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:48.970824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:48.970824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:48.970824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:48.970824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:48.976672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 243/485] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:48.979827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:48.983447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.041276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e78d6c-6aa0-4cbc-890c-e5193a0693c7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.041883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e78d6c-6aa0-4cbc-890c-e5193a0693c7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.042321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2e78d6c-6aa0-4cbc-890c-e5193a0693c7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.046625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 244/486] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:49.053231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fde556-dfca-4fd3-84db-07d3c9469c36 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.054064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fde556-dfca-4fd3-84db-07d3c9469c36 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.054786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fde556-dfca-4fd3-84db-07d3c9469c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.093733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-83fde556-dfca-4fd3-84db-07d3c9469c36 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:49.095298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-83fde556-dfca-4fd3-84db-07d3c9469c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:49.131090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.134349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288b4e9c-47d4-4527-baa0-1f6704a7b735 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.171857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.172865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.173304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.173775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.181210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:49.182077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:49.189097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:49.193325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 245/487] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:49.194653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141050-0381-4431-a032-7f597ff194f5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.195100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141050-0381-4431-a032-7f597ff194f5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.195472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81141050-0381-4431-a032-7f597ff194f5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.197155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 246/488] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:49.202875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.203276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.203691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.209713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 243/489] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:49.211368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.229341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:49.230642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:49.286321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.320677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 247/490] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:49.321960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.356335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae80d16c-eef5-466d-abee-d4af6e96e0e8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.405187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.406452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.406864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.407244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.418101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:49.420112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:49.438256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:49.444599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:49.445681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:09:49.446565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:49.458992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 244/491] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:49.458992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.469933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfe58bb-a63f-4b95-8728-4784989c7940 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.520917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.521655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.521939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.522214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.539372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:49.548923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:49.550462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:49.551389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:49.553863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 248/492] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:49.555060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.590337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-571d861e-e804-4f94-9888-f6b579273b38 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.619920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.634145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.635393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.635925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.636400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.650105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:49.659691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:49.661072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:49.661887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:49.665015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 245/493] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:49.666414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.698592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e58c10d-3503-4892-9b25-4055fe6ca7ca None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.746603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.747820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.748050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.748313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.749258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:49.749567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:49.751100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:49.782033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.798989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de42420-2ac9-4518-8273-fcb3755fce7a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.800590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 249/494] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:49.801593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.839118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.841766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.842248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.842701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.843644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:49.844151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:49.845765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:49.858633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 246/495] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:49.859823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.916072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:49.924187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596af19d-85ea-4e3e-ac2d-114ac539821b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:49.960034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:49.960757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:49.961070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:49.961369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:49.962109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:49.962421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:49.963819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:49.972096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 250/496] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:49.973171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:49.978668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b01e894-29f0-4865-82b2-2061f9d7f8d1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.013723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.014381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.014668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.014949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.036396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.077507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9c2538-d500-47a0-8272-ab85064b58db None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.082258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.083721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.110275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.111088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.111366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.111643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.123620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.125110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.127457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-71e093ab-6fcf-402c-8720-b4e080fc7177 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:50.142624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:50.143642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:50.144828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:50.144828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:50.144828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.144828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:50.144828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 251/497] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:50.145337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4044e-6f4c-439d-a9aa-a7ebee3c48bd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.145575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4044e-6f4c-439d-a9aa-a7ebee3c48bd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.145852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e4044e-6f4c-439d-a9aa-a7ebee3c48bd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.147113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 252/498] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:50.148175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d9ec0-7dfa-4541-9ffb-02be098b09e6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.148798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d9ec0-7dfa-4541-9ffb-02be098b09e6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.148798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900d9ec0-7dfa-4541-9ffb-02be098b09e6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.149943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 253/499] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:50 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:50.150634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d9ec0-7dfa-4541-9ffb-02be098b09e6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.187979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 247/500] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:50.188948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.259351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900d9ec0-7dfa-4541-9ffb-02be098b09e6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.272866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.293491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97862c20-c75b-4f23-8f49-1b12494c9220 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.294502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.295300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.295588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.295862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.307263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.308774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.325994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.326654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.326935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.327207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7e092c81-629f-4f8c-9d29-6bc10b63c3b8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:50.327533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:50.328451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:50.329385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:50.329385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:50.329385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.329385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:50.329589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:50.329589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.329715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 254/501] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:50.329779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c8385e-990c-41f7-a267-b9eafacb310d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.330362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.330656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.330940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.342596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 248/502] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:50.344983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.345354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.345702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.350351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:50.351547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:50.351736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.362493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:50.363937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:50.419908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 255/503] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:50.420955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.440259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 249/504] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:50.441833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.519615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.525894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f94529-1b97-403f-aee8-83a0d61053dd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.542845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d12c83-7af3-4219-a472-931e1bcce1e2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.562872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.563601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.563892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.564165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.575070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.576531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.577465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.578185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.578407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.578682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.579409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:50.579711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.581645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.583297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:50.594457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:50.595400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:50.596242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:50.596242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:50.596242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.596242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:50.596242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 256/505] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:50.596494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.608612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 250/506] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:50.608612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.704850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b074b-2c28-48bb-94bd-8868dc02c091 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.714029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20dc3bf-4f80-4d93-bf0a-d9c2642d25ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.741156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.741973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.742257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.742568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.750713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.751742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.752201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.752578 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.752978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.753259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.765787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.767265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:50.769994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:50.770930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:50.771817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 257/507] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/users/glance => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:50.772381 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.783918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:50.791655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:50.798814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:50.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:50.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:50.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:50.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:50.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 251/508] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:50.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.839896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:50.879037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26049192-9289-415f-94b9-8ecc5a4182c3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.892630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5056d38a-264f-4897-bd96-69a87f476ac7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:50.915031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.915832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.916117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.916395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.927124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:50.932700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:50.933365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:50.933642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:50.933920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:50.934637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:50.934792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:50.935795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:50.936638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:50.937242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:50.937310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 258/509] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:50.937373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:50.937437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:50.949350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 252/510] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:50.950226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.008214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.043343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c08fa14-9a05-48b3-be63-a3b4d034aaa3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.050463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e351774-9989-40ab-8044-8d6565601d20 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.058062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.078450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.079246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.079528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.079798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.084667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.085335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.085617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.085895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.086746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:51.087056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.088348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.088707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.089818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:51.104469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:51.105486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:51.106395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 259/511] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:51.106944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.130464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 253/512] 10.222.0.70 () {56 vars in 1137 bytes} [Mon May 23 13:09:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:51.131398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.222864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fffc4f3-3442-460f-bad8-89c0b2e6dc7c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.235210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f83adb-f422-47ce-9c0d-de80607f3a62 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.249955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.258877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.259542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.262253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.262253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.262253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:51.262253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.262541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.270822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.271499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.271782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.272058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.272838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:51.273148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.274554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.274642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 260/513] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:51.275412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.288068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 254/514] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:51.288068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.312843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.377511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde982d4-d0e5-42c4-87dd-2c6743903881 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.385370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da3ae640-829f-4c52-b779-46d950f47b44 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.413120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.413920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.414204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.414474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.421252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.421637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.422304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.422585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.422694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.422857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.423686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:51.423992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.425461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:51.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:51.438030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:51.438862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 261/515] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:51.439328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.443582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 255/516] 10.222.0.70 () {56 vars in 1137 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:51.444384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.488293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.529699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.546543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4803dad2-e21e-4a30-885c-1582386da9d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.555296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4531350-6132-4beb-a3ce-2ac871fab7da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.587962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.588926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.589209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.589479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.590191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.591635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.596607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.597906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.598145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.598542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.672872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.674268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.678184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 256/517] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:09:51 2022] POST /identity/v3/users => generated 247 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:09:51.679012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.728513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.799177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbc4080-e605-4e2b-84f5-b59b1ba4bd4c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.803750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 262/518] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:51.804462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.841776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.842691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.842981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.843260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.853364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:51.856258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.857818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:51.876816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:51.878228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:51.879853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:51.879853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:51.879853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:51.879853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:51.879853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 257/519] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:51.880123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:51.929730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83ed1745-19cf-4269-8828-0acefc01618a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:51.968596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:51.969610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:51.969895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:51.970174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:51.970967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:51.971277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:51.972770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:51.991724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 263/520] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:51.992605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.005037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.014960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ab3bd0-e300-4044-8795-e10523adef6c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.058725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.060176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.060817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.061318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.104515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05a7346-8898-4afa-a17e-5c30722c8f05 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.130366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.140884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.142361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.142902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.144115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.144115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.144527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.156308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.157805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:52.176749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:52.177641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:52.178494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:52.178494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:52.178494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:52.178494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:52.178494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 264/521] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:52.179138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.245577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.270620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 258/522] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:52.270620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.293349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49fedcc-6175-4fb5-b683-341edf45c82e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.327839 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.328503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.328812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.329089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.329817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:52.330128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.331580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.343227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 265/523] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:52.344041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.379339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.388248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb906d-78c2-4e67-8137-3b12f0a98744 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.427777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.428440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.428760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.429044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.430002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=9cba93afd3344df298ba353d9994ac37) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:52.430320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.431907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.438601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 259/524] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/role_assignments?user.id=9cba93afd3344df298ba353d9994ac37&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:52.446428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.457324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da13cf1-80b3-4eb2-bfc9-9a5bdc57f697 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.479551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.499258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.500119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.500507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.500954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.501835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:52.502251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.504585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.521299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 266/525] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:52.522533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.596908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56444a66-fa71-4ce1-b4a4-fb7e8f0c8a70 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.640713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.641676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.642083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.642452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.656961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.658748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.671799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a101b6c7-a296-4f5d-9936-bb7f0c054c83 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:52.678435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:52.679520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:52.680650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:52.680650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:52.680650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:52.680650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:52.688682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 260/526] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:52.688682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.707946 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.729370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.730732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.731219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.732045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.744229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.746187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:52.766175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:52.767151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 13:09:52.768030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:52.772272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 267/527] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/users/neutron => generated 85 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:52.774617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.837236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:52.840799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59785244-0380-478f-9e4b-1cd6dbcb2c06 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.892632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.893983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.894495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.894495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.902591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.904179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:52.921532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820ab65-8775-414c-a900-07daa51be693 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:52.923425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:52.924624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:52.926194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 261/528] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:52.926649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:52.974666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:52.975450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:52.975924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:52.976240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:52.977107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:52.977423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:52.979736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.000726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 268/529] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:53.000726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.034821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.082868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23eddb-2cc8-423a-9f9a-f7691c30634f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.125936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa324c81-19d3-49f7-981d-40d53aa88419 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.126600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.127545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.128502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.129904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.130994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:53.131513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.133326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.145402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.157236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 262/530] 10.222.0.70 () {56 vars in 1139 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:53.158369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.202711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.203384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.203664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.203931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.204753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:53.205092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.206570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.224846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 269/531] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:53.225032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.342339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cf6416-cb1d-4bb5-8882-2a8ee642b58f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.345839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7584c688-a345-4ebc-82d5-966dc153d79b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.388462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.389170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.389453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.389724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.393042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.413606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.414940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.415546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.416031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.421839 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.427530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.429368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:53.449669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:53.450623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:53.451818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:53.455252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 263/532] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 297 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:53.456535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.473913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.473913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.594626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 270/533] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:53.594626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.635767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdab66d-41d6-4858-b79b-77fecc42be50 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.656127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.703903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.705272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.708093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.708803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.711070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=99a35674d90447d0a7ee255845280be4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:53.711864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.713646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.713766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949de350-6952-4540-b93b-64fc34eeb29d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.729354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 264/534] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/role_assignments?user.id=99a35674d90447d0a7ee255845280be4&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:53.730581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.751813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.752237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.752524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.761860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.761860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:53.761860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.761860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.770662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 271/535] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:53.776773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.806412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.868982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e6a667-bcf9-4600-86f6-270349223632 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.911720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.913164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:53.913750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:53.914243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:53.929513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:53.931459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:53.948261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71300a7e-4877-466c-a857-6bfdb55435d4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:53.953957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:53.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:53.955897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:53.955897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:53.955897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:53.955897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:53.958171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 265/536] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:53.965676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:53.985520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:53.998320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:53.998989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.000347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.000347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.000347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=1ff9d52fc1194828898369d1237b1337) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:54.000716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:54.002220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafcb04a-a3bb-47dd-bd15-7bb0d49aa89a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:54.010157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 272/537] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/role_assignments?user.id=1ff9d52fc1194828898369d1237b1337&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:54.148937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf43a03-3d8c-4e10-bdd9-46282e80b02c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:54.150855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:54.202640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344016e-f6b5-4018-8a66-268d1d6c8cf8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.202949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344016e-f6b5-4018-8a66-268d1d6c8cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.203196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a344016e-f6b5-4018-8a66-268d1d6c8cf8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.204860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 273/538] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:54.210018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:54.210981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.211485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.211988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.212971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:54.213486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:54.215075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:54.223825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11050d0-334b-453f-a463-ee42916d438a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.224125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11050d0-334b-453f-a463-ee42916d438a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.224387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11050d0-334b-453f-a463-ee42916d438a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.229923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 266/539] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:54.235562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:54.251773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e11050d0-334b-453f-a463-ee42916d438a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:54.261250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e11050d0-334b-453f-a463-ee42916d438a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:54.387109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 274/540] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:54.401256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fe23b8-3f40-4841-acec-cfcd9a231d30 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:54.422368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11050d0-334b-453f-a463-ee42916d438a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:54.447644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:54.448453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.448774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.449048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.463041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:54.464506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:54.494917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:54.495836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:54.496700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:54.498315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 267/541] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:54.504463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:54.546906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:54.606639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11050d0-334b-453f-a463-ee42916d438a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:54.635090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:54.675798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:54.676516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.676820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.677096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.696317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2d40f6-ae4d-4b99-a800-60cbc6ab6d87 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:54.699537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:54.713550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:54.714479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:54.715272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 275/542] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:54.720505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:54.773901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:54.774578 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.774859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.775131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.775933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:54.776238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:54.781050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:54.816994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 268/543] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:54.823015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:54.873710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b4757f-ba88-4c94-81ab-4e85e506d38b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:54.928963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:54.930035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:54.930526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:54.931007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:54.931965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:54.932476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:54.934852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:54.952553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 276/544] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:54 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:54.958644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:54.974570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:54.995889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f2f653-5524-4ba8-b80e-e6ee3e45d62d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:55.037434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:55.038406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.038869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.038973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:55.039418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.040524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=4ea47f7631ce4094afd5c0c2787f8969) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:55.041156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:55.042812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:55.049709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 269/545] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:54 2022] GET /identity/v3/role_assignments?user.id=4ea47f7631ce4094afd5c0c2787f8969&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:55.122872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb38674-64d4-423e-9f89-0894cbbdfd6f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:55.162575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:55.163576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.163862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.164135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.175625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:55.178875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-62573d0c-d350-4abf-8d37-20af99882ac1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:55.199361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:55.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:55.201658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:55.201658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:55.201658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:55.201658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:55.201658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 277/546] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:55.210096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:55.231804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecb923-e2c2-4e51-920a-9cec37fdb209 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.232160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecb923-e2c2-4e51-920a-9cec37fdb209 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.232346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ecb923-e2c2-4e51-920a-9cec37fdb209 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.234432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 278/547] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:55.244507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.244829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.245116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.278849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:55.281286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:55.404914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d1f97f-7104-4e1c-8c0b-b2c7fabc357a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:55.423430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 279/548] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:09:55.447318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:55.447911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.448193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.448469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.450108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:55.450421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:55.451263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:55.451849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:55.466282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 270/549] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:55.471844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:55.485213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:55.491513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:55.639956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81154a5a-33d5-4da4-a510-66f003adf1cd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:55.679359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:55.680310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.680625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.680908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.693758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:55.698592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8fd540-b948-47b0-86fe-4292f4cba188 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:55.702988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:55.703909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:55.704819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 280/550] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:55.712006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:55.774648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:55.775463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:55.775747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:55.776023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:55.791308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:55.793973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:55.819428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:55.820438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 13:09:55.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:55.829661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 271/551] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:09:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:55.837434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:55.867609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:55.906561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4932b8f8-5754-468b-959c-53ccb2994f53 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:55.940794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:55.976451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a89612-430d-47a9-9aab-19d6d9628c9d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.005292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.005981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.006268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.006548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.007310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:56.007670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.009104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.013518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.014526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.015011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.015492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.016577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:56.017096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.019475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.036114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 281/552] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:09:56.045579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 272/553] 10.222.0.70 () {56 vars in 1135 bytes} [Mon May 23 13:09:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:56.047415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.051150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.222487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9a0587-43ba-4114-9ac6-11286b524aa5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.273365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.274622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.276419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.277323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.292484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:56.300133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.307497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.317151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981a82cb-5d68-431b-a612-db775afdf71e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:56.340455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:56.341392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:56.342349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:56.342349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:56.342349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:56.342349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:56.345656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 282/554] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:09:56.351968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.370538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:56.393889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8fa055c-c6b5-476c-8bf5-c3eb0d956886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.397988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa055c-c6b5-476c-8bf5-c3eb0d956886 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.398496 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa055c-c6b5-476c-8bf5-c3eb0d956886 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.398996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8fa055c-c6b5-476c-8bf5-c3eb0d956886 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.507700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b539706-aeb4-4093-bf3c-d11aa32acb69 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.515403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa055c-c6b5-476c-8bf5-c3eb0d956886 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.517004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8fa055c-c6b5-476c-8bf5-c3eb0d956886 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.549864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.551082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.551573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.552114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.553082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:56.553591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.555306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73493069-62ce-4724-8c8a-1fb20520dea3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.569828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 283/555] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:56.571615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bfa9e-a423-4015-931c-73384bdedd2f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.572112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bfa9e-a423-4015-931c-73384bdedd2f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.572609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73bfa9e-a423-4015-931c-73384bdedd2f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.574653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 284/556] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:56.575667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bfa9e-a423-4015-931c-73384bdedd2f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.638834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 273/557] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 584 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:09:56.640282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.640595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.640877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.644925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:56.658110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:56.659694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:56.678488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:56.694774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73bfa9e-a423-4015-931c-73384bdedd2f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.732482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 274/558] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:56.732812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.733501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.733788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.734058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.734560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.744877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.746579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a9c33a01-2a8c-475e-a2a0-1214379a3852 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:56.762785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:56.764222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:56.765192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 285/559] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:56.765771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b922c9-3046-4ff8-ba73-ada0544e556d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.765771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b922c9-3046-4ff8-ba73-ada0544e556d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.765899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b922c9-3046-4ff8-ba73-ada0544e556d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.772098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 286/560] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:56 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:56.773093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b922c9-3046-4ff8-ba73-ada0544e556d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.857149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e97afc0-74c7-45eb-88be-69ede671f6d7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.893741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b922c9-3046-4ff8-ba73-ada0544e556d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:56.897345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.898367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.898655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.899201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.908025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:56.911974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:56.913517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:56.933071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:56.934262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:56.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:56.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:56.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:56.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:56.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 275/561] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:56.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:56.939261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4b1738-e46e-4868-ae8d-c7b294ffb747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:56.940768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1738-e46e-4868-ae8d-c7b294ffb747 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.941314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1738-e46e-4868-ae8d-c7b294ffb747 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.941314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4b1738-e46e-4868-ae8d-c7b294ffb747 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.946859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:56.957305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4b1738-e46e-4868-ae8d-c7b294ffb747 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bc4b1738-e46e-4868-ae8d-c7b294ffb747 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:56.965968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:56.974343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 287/562] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:56.975157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:56.975974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:56.975974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:56.993787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:56.995127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:57.067795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db20c37-9677-4c4a-b0b0-ab61800a84e6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.079972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 288/563] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:57.081547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.115544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.116146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.116434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.116751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.117647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:57.117950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.119421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.148623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 276/564] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:09:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:57.149493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.214551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.217775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdea8e0-acb0-49ba-9874-b50260aab87c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.225974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.254641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.255691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.256221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.256845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.257588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:57.257901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.259291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.270742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8e8274-a6bf-4772-9ee2-1714872e1f0f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.271539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 289/565] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:57.272394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.313964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.314634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.314950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.315223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.316088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:57.316396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.317932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.326746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 277/566] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:57.326746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.390505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b658c4f7-cdaa-4014-8710-101675570ed8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.428182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.429041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.429318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.429587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.437399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2eda862-b6a8-4218-83eb-e046dff57fcc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.444442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:57.452430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:57.453724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:57.454604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 290/567] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/roles/service => generated 85 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:57.455296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.474497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.480207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.485313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.486121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.486416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.486950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.498245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.499718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:57.518379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:57.519918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:57.525869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:57.525869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:57.525869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:57.525869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:57.525869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 278/568] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:57.525869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.569158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db2133ac-c1ad-4320-b7c5-b494d6ef3218 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.608697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.609599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.609952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.610295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.619068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.620528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:57.636050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:57.637273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:57.642289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:57.642736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f72ebb6-6c21-4c63-b0bd-30f33f71702a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.643337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 291/569] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:57.643337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.675768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.676581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.676877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.677153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.687367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.688853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:57.707474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:57.708597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:57.709868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:57.709868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:57.709868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:57.709868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:57.709868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 279/570] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:57.709868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.728043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.745504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.749850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e04e9ac-bc8f-449d-bb79-79dececa2d18 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.784698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.785372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.785649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.785919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.786660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:57.786968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.788325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c7f776-4252-4402-aae2-f68f80d8b509 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.796208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 292/571] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:57.797560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ff59-b47e-4db7-8da2-8e847dedf590 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.797846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ff59-b47e-4db7-8da2-8e847dedf590 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.798121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b067ff59-b47e-4db7-8da2-8e847dedf590 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.799937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 293/572] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:09:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:09:57.800712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ff59-b47e-4db7-8da2-8e847dedf590 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.820864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168d512b-bbfa-4c88-ad86-413c4d76b43d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.858078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.858864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.859210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.859493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.860222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:57.860531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.862571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.874546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 280/573] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:57.875378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.907438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b067ff59-b47e-4db7-8da2-8e847dedf590 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:57.943641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:57.944302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:57.944612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:57.944918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:57.945721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:57.946031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:57.947452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:57.959508 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 294/574] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:57.960341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:57.963014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.979017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:57.989014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0ee318-1798-4a98-990a-8cb9b8407496 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.022735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.023752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.024051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.024330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.025097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:58.025409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.027620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa54a76-2f34-420d-95df-d294c4da5580 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.041084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 281/575] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:58.042448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.042741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.043019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.060199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:09:58.061422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:09:58.072236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbd208-8616-4097-bc5e-fec55dc61c9f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.106920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.107716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.108269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.118865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.120308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.129532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 282/576] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:09:58.130384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:58.138747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:58.139786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:58.140878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:58.140878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:58.140878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:58.140878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:58.142729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 295/577] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:58.144114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.203110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.218609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.237261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599744b-2c3c-4f92-a9af-c5dd73a3b71d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.247143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17369acd-2eba-472b-b25b-d55dd6cd8959 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.270651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.271441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.271727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.271995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.278890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.280437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.281719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.282414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.282689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.282962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.283814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=e525e14157f94459a6e2ab46ffdeb529) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:58.284120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.285560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.290959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 296/578] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/role_assignments?user.id=e525e14157f94459a6e2ab46ffdeb529&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:58.291750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:58.294961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:58.296030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 283/579] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:58.297124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.417020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1310103-933f-445c-bbf9-3f10c6bdff1a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.417490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4fcd56-112b-4bf6-beb7-afcb617af2b5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.437920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.456769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.457467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.457757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.457932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.458052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.458716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.458814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:58.458996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.459102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.459269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.460486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.467973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.469483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.472237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.477043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 284/580] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:58.477655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:58.487702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:58.491805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 297/581] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/users/glance => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:58.492801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.589585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c8b85b-8cc1-4531-948a-be6898d13813 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.606224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160b7b37-1141-4650-aff5-a8ea573ad98e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.632167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.633287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.633577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.633877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.642584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.643318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.643666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.643987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.644819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:58.645309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.646759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.648715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:09:58.659478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:58.660421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:09:58.661278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:09:58.662335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 298/582] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:58.662335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.663822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 285/583] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:58.664847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.714150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.735842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.778314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396b52a4-e693-4cf5-ad34-ab4b06aa4af9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.789121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3078d079-f967-4a25-a7be-83c723b3df1f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.819211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.819901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.820190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.820472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.821362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:58.821709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.823162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.831658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:58.832463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:58.832773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:58.833049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:58.840642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:58.842098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:58.848127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 286/584] 10.222.0.70 () {56 vars in 1137 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:58.848127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:09:58.862938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:58.864111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 299/585] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/users/cinder => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:58.865122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:58.964703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8fe717-2540-444f-9ec0-67a39783746e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.977982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:58.982696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d63474-c1fd-418e-8469-2ccf3f0c6b3f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:58.984461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:59.001511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.002515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.002915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.003259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/9cba93afd3344df298ba353d9994ac37/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.017556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.019185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.019455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.019592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=9cba93afd3344df298ba353d9994ac37, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.019865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.020734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:59.025553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.025705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.025816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.033086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 300/586] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:09:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:59.034290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.102198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 287/587] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:09:58 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/9cba93afd3344df298ba353d9994ac37/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:09:59.112480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.157272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83742d3-15a3-4594-9525-ef36b3a6ec4c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.198731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.200165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.200165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.200399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.213785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.215635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:59.236175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:59.237353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:59.238421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:59.238421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:59.238421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:59.238421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:59.244629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 301/588] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:59.245729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.264393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:59.272287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90fa5646-850a-41d9-96fb-bb7f56e6732c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.313183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.314009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.314365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.315167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.316021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:59.316328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.318711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.337992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 288/589] 10.222.0.70 () {56 vars in 1137 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:59.338909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.372377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:59.382697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9975c6-cc08-4daa-ac85-4a0de1c7dd55 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.435835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.436677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.436962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.437239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.449478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.451052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:59.472354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:59.473464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:59.474461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:59.474461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:59.474461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:59.474461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:59.474918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fbee66c-2825-4f9c-badf-05d41a01ab69 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.475086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 302/590] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:59.479472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.522138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.523267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.523549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.523823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.525197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:59.525646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.527780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.545017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 289/591] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:59.549014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:59.549734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.592670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0331212c-c078-4eef-8cc8-cab31e4aa4db None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.640136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.641808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.642327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.642861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.655486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.657609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.661343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:59.677914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa93894-785a-4052-93f5-d0d25bab1c86 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:09:59.684292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:59.685179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:59.686293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:09:59.686293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:09:59.686293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:09:59.686293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:59.686293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 303/592] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:59.687007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.717405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.718133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.719183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.719740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.721215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:59.722221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.723767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.744301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 290/593] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:59.745341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.792605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea926f3-df99-4a6a-b5ac-b14cce9041da None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.814616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:09:59.841831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.842803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.843158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.843530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.853623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.856164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.865291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfeeb08-3148-4cec-9b1f-97536af263b9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:09:59.881338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:09:59.882266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:09:59.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 304/594] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:09:59.884258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.907182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:09:59.907850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:09:59.908129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:09:59.908556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:09:59.909413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:09:59.909749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:09:59.911176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:09:59.930991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 291/595] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:09:59.931970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:09:59.944471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.003492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc8a560-7dd2-4734-86bd-9477bb38dbae None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.043711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.045045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.045943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.046379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.057477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.059730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.066604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9d4841-92fb-4e8c-892e-9aef724a0eb0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:00.079931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:00.081209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 13:10:00.082347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:00.090076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 305/596] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/users/neutron => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:00.090076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.111056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.112336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.113036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.113148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.114289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:00.114740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.116287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.135787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 292/597] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:09:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:00.136985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.137721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.222219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b07ed38-1501-40cd-96a6-027ace25ab78 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.229025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.260730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.261459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.261745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.262024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.269188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-846aa74e-a09b-4ffb-a1af-917ac1abaa58 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.269358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.270776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:00.287395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:00.288333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:00.289344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 306/598] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:00.289823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.311791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.312898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.313431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.313925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.315151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:00.315669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.317431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.340529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 293/599] 10.222.0.70 () {56 vars in 1139 bytes} [Mon May 23 13:10:00 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:00.341736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.413191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.422323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6dec9b-52c8-4376-a346-b1f1b8220301 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.473055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.474996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.475724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.476179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/99a35674d90447d0a7ee255845280be4/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.477767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd74497-d9e0-45d1-b059-e184f449cf15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.493300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=99a35674d90447d0a7ee255845280be4, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.495153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.504160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.514812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.515420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.515734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.516045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.516892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:00.517204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.519343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.531722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 294/600] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:00.532370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.568597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 307/601] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:00 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/99a35674d90447d0a7ee255845280be4/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:10:00.570083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.675821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382fb6ed-757e-437c-ade3-b27dc8e4405b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.716356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08d707-69d1-4f65-b73d-40298755228d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.722205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.723209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.723560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.723907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.745501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.748005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.760173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-876c9028-bb1a-419b-964f-05cf395b6e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.761186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9028-bb1a-419b-964f-05cf395b6e97 None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.761473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9028-bb1a-419b-964f-05cf395b6e97 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.761752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876c9028-bb1a-419b-964f-05cf395b6e97 None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/1ff9d52fc1194828898369d1237b1337/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5d0627f2-f3e6-419a-bd78-c3e4d7ce7518 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:00.770845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:00.771938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:00.773081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:00.773081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:00.773081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:00.773081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:00.777040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 295/602] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:00.777040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0639d-fe23-4e78-980d-2d7c28643908 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.777040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0639d-fe23-4e78-980d-2d7c28643908 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.777040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f0639d-fe23-4e78-980d-2d7c28643908 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.779674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 296/603] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:00.779674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0639d-fe23-4e78-980d-2d7c28643908 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:00.783991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c9028-bb1a-419b-964f-05cf395b6e97 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=1ff9d52fc1194828898369d1237b1337, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.785917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-876c9028-bb1a-419b-964f-05cf395b6e97 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.791892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.823653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:00.895801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 308/604] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:00 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/1ff9d52fc1194828898369d1237b1337/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:10:00.896419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f0639d-fe23-4e78-980d-2d7c28643908 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:00.897596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.897981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.898414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.924860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:00.927712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:00.950583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:00.951585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:00.952222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:00.953079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:00.954070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:00.954586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:00.957701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:00.987954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 297/605] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:00.992931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.063298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 309/606] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:01.091061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.169013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:01.173042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1935cea0-9cc0-4ccf-8137-17ec6f4d8459 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:01.218992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:01.219861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.220436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.220436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.229169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:01.230750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:01.246933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:01.247895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:01.248820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 298/607] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:01.257283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.278467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390284f3-8047-4f96-8f19-7dbe7d52ce9c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:01.335383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:01.368725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:01.370032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.370532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.371061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.402302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0c6c74-89b3-436d-a52e-85572ae30c87 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:01.417243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:01.426089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:01.427025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:01.427842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:01.430566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 310/608] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:01 2022] GET /identity/v3/roles/service => generated 85 bytes in 340 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:01.436253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.473311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:01.474000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.474292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.474573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.475390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:01.475702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:01.477267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:01.506874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 299/609] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:01.514191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.547676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:01.580735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc2b68c-8201-4473-aefd-84e99f1f720e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:01.631886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:01.633746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.634036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.634317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.635141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:01.635391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:01.636799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9285d922-e734-436f-b399-c3d7effa354d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:01.644511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf9511c-a8b8-4c44-94cb-9b27cccc9449 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:01.646276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 311/610] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:01 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:01.648469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2023d-60cb-455a-875f-32b1fcb3ffca None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.648469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2023d-60cb-455a-875f-32b1fcb3ffca None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.648704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e2023d-60cb-455a-875f-32b1fcb3ffca None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.654681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 312/611] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:01.655552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2023d-60cb-455a-875f-32b1fcb3ffca None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.684529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a6993a7-ecf4-4d6c-82f7-eb5290b93707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:01.686166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6993a7-ecf4-4d6c-82f7-eb5290b93707 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.686689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6993a7-ecf4-4d6c-82f7-eb5290b93707 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.687216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a6993a7-ecf4-4d6c-82f7-eb5290b93707 None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/4ea47f7631ce4094afd5c0c2787f8969/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.703618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6993a7-ecf4-4d6c-82f7-eb5290b93707 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=4ea47f7631ce4094afd5c0c2787f8969, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:01.705684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a6993a7-ecf4-4d6c-82f7-eb5290b93707 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:01.709178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:01.760391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e2023d-60cb-455a-875f-32b1fcb3ffca None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:01.800458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 300/612] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:01 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/4ea47f7631ce4094afd5c0c2787f8969/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:10:01.802820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.803396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.804156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.805278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:01.806222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:01.806511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:01.806834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:01.837982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:01.842023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:01.845504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:01.848281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:01.883876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:01.884832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:01.885751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:01.885751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:01.885751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:01.885751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:01.888275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 313/613] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:01.893903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:01.995979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.011861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 301/614] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:02.056122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.078059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.082317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3004ba0-0569-48bc-aeb9-648f2511aa20 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.120089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.120899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.121183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.121456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.122185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:02.122492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:02.125094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:02.138603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 314/615] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:02.143875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.166639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd540c5-62fc-408c-b485-7050013a1f3a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.204259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.205497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.205832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.206113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.220387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:02.239072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:02.240077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:02.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 302/616] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:02.254479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.271768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149826c-06ad-49f4-aee4-8eab6a93aaca None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.311810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.312749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.313036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.313315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.323198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:02.324680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:02.340267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:02.342516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:02.344801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:02.346666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 315/617] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/users/swift => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:02.352773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.353036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.399301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142fd1c0-5eb8-48a0-91b0-6418a3c1bf76 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.409157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.435794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.436488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.437263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.437263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.437840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:02.438158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:02.440746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:02.449956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 303/618] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:02.456764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.558253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d2285e-1a6d-4368-8a1d-5297de02a855 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.569295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8c0a50-4f3f-4239-90ae-7ac38543cbe5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.601026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.611060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.614496 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.614788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.615070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.634395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:02.635923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:02.646054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.646746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.647035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.647312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.648170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:02.648481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:02.649997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:02.656140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:02.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:02.658406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:02.658406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:02.658406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:02.658406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:02.658406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 304/619] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:02.664845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.699897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 316/620] 10.222.0.70 () {56 vars in 1135 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 352 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:02.706980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:02.881592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf7ea9-4b91-4049-bdaf-6c6c80ceb03b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.885492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:02.930828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db9c571-e98c-44b9-a949-b359d7fab18d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:02.954365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:02.955898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:02.956673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:02.956961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:02.957716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:02.958030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:02.959447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:02.989089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 305/621] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:02.995781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:03.001549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.003448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.003736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.004019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.027688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.029214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:03.063311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:03.066586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:03.070664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:03.070664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:03.070664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:03.070664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:03.079531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:03.079789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 317/622] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 360 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:03.079789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:03.155014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568abc39-e2fa-4b91-9efa-d9b26b8de457 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:03.218188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.219435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.219922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.220400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.238044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.240723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:03.277615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:03.279054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:03.285403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 306/623] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:10:02 2022] GET /identity/v3/users/nova => generated 82 bytes in 284 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:03.293582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:03.294164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246ce194-24c6-4798-b48a-5df29b8f1155 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:03.348057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:03.361354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.365613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.365613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.365613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.365613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:03.369678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.371787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.401961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 318/624] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:03.408729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:03.498161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:03.513297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f18a90c-a774-4ea1-ab25-76bd0efda348 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:03.565450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1a6201-59a6-4025-8fe5-2cdcd4ec9508 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:03.567314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.568361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.568784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.569166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.570392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:03.570828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.573066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588301a-0699-4ec2-b0d0-c5fe4693e706 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.601649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 307/625] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:10:03 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:03.603076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fbff4-2a83-440f-9ef5-11cd23ad6963 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.603374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fbff4-2a83-440f-9ef5-11cd23ad6963 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.603666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5fbff4-2a83-440f-9ef5-11cd23ad6963 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.605188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 308/626] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:03.610571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bee7f-b450-483d-8924-9cacff585759 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.610872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bee7f-b450-483d-8924-9cacff585759 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.611214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0bee7f-b450-483d-8924-9cacff585759 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.619964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.621432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.621834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.622219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.630086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6d0bee7f-b450-483d-8924-9cacff585759 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:03.631398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6d0bee7f-b450-483d-8924-9cacff585759 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:03.632361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.634677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:03.659300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:03.661550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 319/627] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:03 2022] GET /identity/v3/projects/service => generated 88 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:03.663022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:03.718973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 309/628] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:10:03.722233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf63cc-8438-45e8-a329-06b8cab669e0 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.722517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf63cc-8438-45e8-a329-06b8cab669e0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.722801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaf63cc-8438-45e8-a329-06b8cab669e0 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.724376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 310/629] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:03.731030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf63cc-8438-45e8-a329-06b8cab669e0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:03.739566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:03.790203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1c678d-819e-4665-af11-d954ba7c0ffa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:03.834490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.835148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.835438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.835716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.838608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:03.864003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaf63cc-8438-45e8-a329-06b8cab669e0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:03.910409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:03.911340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.911739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.912191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.913483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:03.913951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.915051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:03.915882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a86bc7-72fa-49ee-9adc-6e35cf69f51b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.916505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:03.939755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 311/630] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:03.939755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:03.939755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:03.939755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:03.961679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:03.963361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:03.975653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.036361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 320/631] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:04.037560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.056685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 312/632] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:04.056685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.136459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.168278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e418a2c-1d28-4ade-a85f-066406e2991a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.175651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c06786f-0f97-4517-bcff-7ffa2a378bf1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.224157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.225670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.226175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.226628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/e525e14157f94459a6e2ab46ffdeb529/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.244547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=e525e14157f94459a6e2ab46ffdeb529, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:04.246068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:04.250093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.250874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.251147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.251424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.262393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:04.270708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:04.271677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:04.272540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 321/633] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:04.273282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.289503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.332766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 313/634] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:04 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/e525e14157f94459a6e2ab46ffdeb529/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:10:04.333967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.439858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-114d834b-6ffb-4f4c-aa3c-9fc24ce4bca2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.461117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0804e7c3-5030-433b-9380-2d1f6ba82175 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.484762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.487163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.488111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.488395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.489314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.502193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:04.503697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:04.505330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da7b5a4-624a-4e66-b88e-558219063075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.506674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7b5a4-624a-4e66-b88e-558219063075 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.507025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7b5a4-624a-4e66-b88e-558219063075 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.507310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da7b5a4-624a-4e66-b88e-558219063075 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.521444 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da7b5a4-624a-4e66-b88e-558219063075 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:04.529947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:04.537153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:04.538100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:04.538100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:04.538100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:04.538100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:04.538100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 322/635] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:04.538100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5da7b5a4-624a-4e66-b88e-558219063075 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:04.538426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:04.539380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 314/636] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:04.540295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cc28d-0498-46b6-9696-3f5818dcb101 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.544195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cc28d-0498-46b6-9696-3f5818dcb101 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.544195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906cc28d-0498-46b6-9696-3f5818dcb101 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.544195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 315/637] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:04.544195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cc28d-0498-46b6-9696-3f5818dcb101 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.616321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.667510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca184eda-d4e4-4a9d-ac38-42e239057d4e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.672839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906cc28d-0498-46b6-9696-3f5818dcb101 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.706012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.706698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.706983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.707258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.708026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:04.708326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:04.709763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:04.717034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.717712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.717997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.718274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.719094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 323/638] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:04.719255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:04.719429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:04.720913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5cab5b9-6779-4e55-b74a-f6d92dd51e82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:04.724119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.735084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 316/639] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:04.736753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.737210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.737655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.756905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:04.758104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:04.792913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.838437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 317/640] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:04.839336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.864755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c2b9c-4a95-4644-96bb-34b0d64bfabd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:04.899239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:04.910716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:04.911392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:04.913440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:04.913440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:04.913440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:04.913440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:04.914393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:04.929240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 324/641] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:04.931404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:04.960556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10bfd14-3dbc-4d34-bdf6-40680acb28a3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.003011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.003873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.004161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.004441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.015677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.017203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:05.037461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:05.038600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:05.039506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:05.039506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:05.039506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:05.039506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:05.039506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 318/642] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:05.039506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.049999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5118ac83-6e29-4d9d-86d7-c0f5f3a97815 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.068394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.088606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.089420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.094623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.094623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.098032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.099488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:05.114920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:05.115980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:05.116944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 325/643] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:05.117445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.161179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84883fce-e206-476f-81ce-5d0783feda15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.170138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.219914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.221306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.221585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.221852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.237577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c0a4a1-2edf-4600-b9fd-8497a76c6ea0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.238849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:05.247034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:05.248037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:05.248963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 319/644] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:05.249752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.278749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.279550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.279830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.280105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.292560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.294077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:05.312422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:05.313402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:05.314961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:05.314961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:05.314961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:05.314961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:05.314961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 326/645] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:05.314961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.352110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.371939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe906ae0-f8be-4b65-a0ea-54b768d1c39b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.410103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.410605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.410918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.411254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.411927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:05.412233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.413644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.427281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 320/646] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:05.428107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.432441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.437161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cf4fb2-b45a-43e4-9c9f-f805e471807a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.475200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.475970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.477437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.477713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.478503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:05.478812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.480232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.492858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 327/647] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:05.498166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.554720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7670a2a0-0939-4f40-906a-151c8057f4ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.605199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.605873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.606153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.606423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.607169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:05.607474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.608881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.622175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 321/648] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:05.622175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.622175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.641007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226b98b2-89ff-4f76-950e-30ce32afee29 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.685003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.685003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.685003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.685003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.685420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:05.685420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.686047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.700906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 328/649] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:05.706669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.726124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.734259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828625e-9c2e-4670-8ed6-8aa5561672da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.771227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.772027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.772311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.772602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.781689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.784471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:05.801198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:05.802208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 23 13:10:05.803168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:05.804400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 322/650] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/users/glance => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:05.805300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.840261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fd9870-0bff-40dc-a99d-0489385264eb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.868083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:05.890412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.895359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.895359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.895359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.895359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=9cba93afd3344df298ba353d9994ac37) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:05.895359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.897497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:05.907940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 329/651] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/role_assignments?user.id=9cba93afd3344df298ba353d9994ac37&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:05.909052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e29d4e-1802-49e6-8058-f5841c7b1525 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:05.909688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:05.955331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:05.956557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:05.957072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:05.957499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:05.971219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:05.972867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:06.000188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:06.001322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:06.002796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:06.002796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:06.002796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:06.002796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:06.008553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 323/652] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:06.009998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.011561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.038975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b328df9d-43a6-4ad8-b8f1-28c030c5fb55 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.091432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.093066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.093552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.093985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.105433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.107933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:06.130102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:06.131292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:10:06.132527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:06.135504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 330/653] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:05 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:06.137108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.168824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b33cef-708f-4c88-a1c9-9b080e51493c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.196693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.219618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.220762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.221060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.221342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.222200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:06.222850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.223966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.247919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 324/654] 10.222.0.70 () {56 vars in 1137 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:06.248927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.271602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51312e84-4741-4156-a6ae-49baf0740161 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.314877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.315648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.315937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.316217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.316998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:06.317435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.318986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.336881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 331/655] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:06.337889 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.344978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.393660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9dadff3-a81f-4698-b36b-3a8ffcfb7b8f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.442833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.444255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.444738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.445569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.446924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:06.447411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.449901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.475964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc223d50-53b4-4eab-bd3a-2e0f3e5664e8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.478659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 325/656] 10.222.0.70 () {56 vars in 1137 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:06.479840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.511452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.524035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.525330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.525772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.526444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.601157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc36b9e-c395-4ea9-aef1-494e183f9353 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.613080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.614406 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.616788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.648092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73650163-39ac-412f-865e-33174c3718aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.649456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73650163-39ac-412f-865e-33174c3718aa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.649894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73650163-39ac-412f-865e-33174c3718aa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.650315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73650163-39ac-412f-865e-33174c3718aa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.660557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73650163-39ac-412f-865e-33174c3718aa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.662893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73650163-39ac-412f-865e-33174c3718aa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-73650163-39ac-412f-865e-33174c3718aa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:06.682357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:06.683567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 23 13:10:06.684737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:06.692425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 326/657] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/users/neutron => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:06.692425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73650163-39ac-412f-865e-33174c3718aa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.739209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 332/658] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:06.740890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:06.792962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.804963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73650163-39ac-412f-865e-33174c3718aa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.850126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e8c9a-c4b8-4ea2-b969-7ae58d9e2271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.851587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e8c9a-c4b8-4ea2-b969-7ae58d9e2271 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.852094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e8c9a-c4b8-4ea2-b969-7ae58d9e2271 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.852763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e8c9a-c4b8-4ea2-b969-7ae58d9e2271 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.875035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60292ca8-d7ff-457c-b235-e9915c5a6d43 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:06.877405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:06.921096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:06.922090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:06.922576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:06.924067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:06.925708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:06.926209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.928385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.941779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e8c9a-c4b8-4ea2-b969-7ae58d9e2271 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:06.943043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e8c9a-c4b8-4ea2-b969-7ae58d9e2271 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:06.952676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 333/659] 10.222.0.70 () {56 vars in 1139 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:06.953777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.066938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bfd1a7-422c-4f35-9f04-571a8e6eea1e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.107736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.108528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.108838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.109110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.117633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.120616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.122072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:07.140402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:07.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:07.142263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:07.142263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:07.142263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:07.142263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:07.144921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 334/660] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:07.146057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.147747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 327/661] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 458 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:07.154175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdf5e9-abd7-4d8f-9f22-f4eab8371680 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.154461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdf5e9-abd7-4d8f-9f22-f4eab8371680 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.154740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acdf5e9-abd7-4d8f-9f22-f4eab8371680 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.156340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 328/662] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:07.157572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdf5e9-abd7-4d8f-9f22-f4eab8371680 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.165757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.296380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafdf9d-8782-4c13-9a0e-6d919d31cc4a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.311055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acdf5e9-abd7-4d8f-9f22-f4eab8371680 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.344631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf54539b-a87f-408e-a5ea-394af4190880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.345296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54539b-a87f-408e-a5ea-394af4190880 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.345978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54539b-a87f-408e-a5ea-394af4190880 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.345978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf54539b-a87f-408e-a5ea-394af4190880 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.355822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.356495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.356813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.357092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.357816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:07.358122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.359500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.376311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 329/663] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:07.376311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.431127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54539b-a87f-408e-a5ea-394af4190880 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.432819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf54539b-a87f-408e-a5ea-394af4190880 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.440122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.449091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.500061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4be078b-a5da-4684-826e-2c0c3d6b68c1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.536649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.537460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.537798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.538019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.541503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 335/664] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:07.542999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.543464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.543934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.558295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.560024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.561471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:07.562708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:07.586462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:07.587431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:07.588324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:07.588324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:07.588324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:07.588324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:07.588324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 330/665] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:07.588652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.644616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 336/666] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:07.645832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.715015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.730771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc1e64d-668e-4421-bb66-3e894716f155 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.732692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.771273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.772074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.772359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.772806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.780842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.782304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.785821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa756bf3-2fec-4201-8d8f-8c7ba93075c2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:07.797070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:07.798027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 331/667] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:07.803907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.838799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.839610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.839898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.840178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.851141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.852627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:07.871575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:07.872545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:07.873959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:07.873959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:07.873959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:07.873959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:07.877832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 337/668] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:07.878666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.921565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef2bcd-5dce-4a03-b7c1-bb03a7d6c4ca None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:07.959814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:07.960926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:07.961385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:07.961814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:07.962740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:07.963192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:07.964840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:07.978325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:07.979245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 332/669] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:07.979995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:07.991430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3b930d-29c1-4aef-a731-563a79627635 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.009853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.036267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.037105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.037390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.037666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.050492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:08.058247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:08.059222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:08.060124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 338/670] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:08.061335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.104209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc0dd2-81fc-44b6-b932-93bdc67dcdd0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.145601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.146997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.147282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.147559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.148359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:08.148697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.150140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.168401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 333/671] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:08.169376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.191392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0580fb40-adbb-4d16-b83b-a741a89341a1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.231874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.232732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.233024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.233306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.234087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:08.234389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.236419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.255322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 339/672] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:08.255322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.274157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.276467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.295749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285f5c61-fa9d-494e-ac9d-23cdbd9d9d58 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.335902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.337016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.337311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.337711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.345933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.347478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:08.362360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:08.363332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:08.364239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 334/673] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:08.365220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.379901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38478f77-8474-4faa-a1c1-e77473d4e2ed None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.417817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.418499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.418782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.419057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.419892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:08.420117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.421503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.430540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 340/674] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:08.431402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.477888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb192c0-1edc-49e9-9301-e8f8bcb3b235 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.510040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.512956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.513621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.513912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.514187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.515017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=99a35674d90447d0a7ee255845280be4) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:08.515326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.516800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.523160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 335/675] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/role_assignments?user.id=99a35674d90447d0a7ee255845280be4&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:08.523973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.533917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be46f79-7255-4d95-a1d1-ca6e8e2d393c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.545914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.574756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.575582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.575870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.576878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.585057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.586498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3a0869b4-ce25-4347-a3ee-b49f1704aaed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:08.603543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:08.604762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:08.605670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 341/676] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:08.607209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95140-224b-4c56-bdc3-e098b4d3e5fd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.607209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95140-224b-4c56-bdc3-e098b4d3e5fd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.607403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b95140-224b-4c56-bdc3-e098b4d3e5fd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.610284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 342/677] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:08 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:08.612607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95140-224b-4c56-bdc3-e098b4d3e5fd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.668121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf61422-d17a-4941-970c-4bc74e1d29d6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.715443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.716537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.717102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.717620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.718639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:08.719139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.721281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.734878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b95140-224b-4c56-bdc3-e098b4d3e5fd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.736774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 336/678] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:08.737948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.780023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.780895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.781184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.781462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.785876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.794974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.796440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a236cd7f-38c4-4754-8117-6aded8a3b024 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:08.817814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:08.818853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:08.819801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:08.819801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:08.819801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:08.819801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:08.819801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 343/679] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:08.821341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.821341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.821521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.828282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:08.840030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:08.841285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:08.869068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a33b0ec-4518-458d-9def-717763268847 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:08.907005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:08.907893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:08.908282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:08.908560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:08.909387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:08.909762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:08.916719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 344/680] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:08.916877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:08.916958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:08.929589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 337/681] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:08.930662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.053024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749e1fa4-80f1-4ebf-be2b-2a614969e8eb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.066735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7ffa4-dd05-4902-9b96-6d840f3fd41d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.073218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.091546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.092432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.092993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.093286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.093566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.094413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=1ff9d52fc1194828898369d1237b1337) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:09.094727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.097049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.102050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.102722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.103002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.108698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.108910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 345/682] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/role_assignments?user.id=1ff9d52fc1194828898369d1237b1337&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:09.108995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:09.108995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.109130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.109200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.124002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 338/683] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:09.125136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.260166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdaded55-e11e-4519-af3a-de1a0f9b5945 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.271863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bb4eda-0aea-4bd8-8047-6fb658087293 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.305013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.305994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.306455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.306915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.307935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=4ea47f7631ce4094afd5c0c2787f8969) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:09.308426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.310236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.318474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 346/684] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/role_assignments?user.id=4ea47f7631ce4094afd5c0c2787f8969&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:09.319675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.343962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.345931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.346473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.346978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.365918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:09.375474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:09.377324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:09.378202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 339/685] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:09.386428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.407429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.473428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.498348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52925fc0-63c6-42d9-a8e2-03504118c0ac None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.553311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38695a99-6198-4896-9220-bf7a1444e4e3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.560311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.562462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.562980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.563858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.574689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.577220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.590906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.591633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.591916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.592189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.593011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:09.593317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.594697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:09.599088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:09.600027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:09.601013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:09.603499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 347/686] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/users/swift => generated 83 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:09.608730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.611790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 340/687] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:09.620321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.737047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568a1b01-87ab-4ccf-b41c-40091e35f193 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.748503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.781098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.781786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.782069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.782345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.783790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:09.784147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.785846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.807190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 348/688] 10.222.0.70 () {56 vars in 1135 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:09.813008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:09.897438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbf7a79-c910-419e-80ae-35ad0d3bcd1d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.910103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:09.931040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76014342-b0c9-4684-9969-89de006bbcf8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:09.938074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.939610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.940055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.940635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.952752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:09.954588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:09.971687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:09.972342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:09.972853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:09.973194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:09.976680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:09.977778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:09.978853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:09.978853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:09.978853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:09.978853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:09.980366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 341/689] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 361 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:09.989947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:10.010347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:10.048171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:10.050698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:10.116667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64595c3-1b30-4616-85de-421ba5b9f41b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:10.156235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 349/690] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:10.156951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:10.158189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:10.158486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:10.158747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:10.160069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:10.160275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:10.166279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:10.168928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:10.214674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 342/691] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:10.220283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:10.312953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:10.355911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:10.452015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a84bcf-561a-450a-958d-4e59bb345139 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:10.502353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b9fb8c-b132-4040-9df9-dcb673c76a7f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:10.558537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:10.559942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:10.560460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:10.561003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:10.593693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:10.596416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:10.615593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:10.616596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:10.617584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:10.617584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:10.617584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:10.617584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:10.617584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 350/692] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 449 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:10.628479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:10.634018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:10.635287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:10.635758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:10.636217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:10.650372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:10.653457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:10.680413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:10.681455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:10.682406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:10.685038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 343/693] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:10:10 2022] GET /identity/v3/users/nova => generated 82 bytes in 466 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:10.690790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:10.811919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:10.839538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df8a76-15c1-4753-8740-f720a70346a8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:10.893844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df210d58-8867-4c52-82f6-a8e8286f66ed None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:10.898354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:10.899246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:10.899699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:10.900158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:10.901776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:10.902372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:10.904073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:10.930231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:10.934368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 344/694] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:10:10 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:10.940863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:10.966749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:10.967788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:10.968279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:10.968798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:10.970903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:10.972389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:10.974301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.000483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 351/695] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 372 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:11.012554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:11.107217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926e4cd8-1a30-490a-bbda-a6ecadf76947 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:11.155597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:11.156302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.156605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.156886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.167207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:11.234045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:11.235979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.302546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99019a9-aa01-4942-ac56-4b84511f3b81 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:11.341014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 345/696] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 400 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:11.347786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:11.381352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:11.382407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.382702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.383404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.393896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:11.396280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.401343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ff112df7-9ac2-406d-89fd-37f11d9285be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:11.416492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:11.417452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:11.418329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:11.419276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 352/697] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 411 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:11.425693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e56c6-836f-4227-b897-f5a514b01a78 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.426357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e56c6-836f-4227-b897-f5a514b01a78 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.426794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02e56c6-836f-4227-b897-f5a514b01a78 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.430283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 353/698] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:11 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:11.431342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e56c6-836f-4227-b897-f5a514b01a78 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:11.482285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:11.483832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7039e3f4-2666-4ebf-b158-0b417c462c56 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:11.528332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:11.529727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.530155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.530724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.544054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:11.545971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.568915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02e56c6-836f-4227-b897-f5a514b01a78 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ac39d1c7-d4c1-468c-9c21-9f26d3b25e23 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:11.569285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:11.570307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:11.571497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:11.571497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:11.571497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:11.571497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:11.574488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 346/699] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:11.575972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.576408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.576867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.604238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:11.608019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:11.623924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:11.624903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.625280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.625614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.627729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:11.628049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:11.630152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.652707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 354/700] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:11.653733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:11.688340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 347/701] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:11.693647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:11.733046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:11.779546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:11.789690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5050a433-3f8d-4a3e-833c-6c36de24ec7c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:11.829591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:11.830425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.831021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.831150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.831985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:11.832410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:11.833915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.841871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7cc4cab-1ab6-4772-ad7e-af68aa89b153 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:11.846608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 355/702] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:11.847559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:11.884178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:11.885027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:11.885405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:11.885779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:11.886901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=e525e14157f94459a6e2ab46ffdeb529) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:11.887223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:11.888886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:11.901394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 348/703] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/role_assignments?user.id=e525e14157f94459a6e2ab46ffdeb529&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:11.904288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.046465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a232f-75c4-4680-b232-2342708192a9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.096898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:12.129558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7372f65d-82a1-468e-94cd-920b986c82b8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.132903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.133798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.134092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.134438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.146647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.148110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5b1e1f6e-56a9-4909-a714-d6ba9bc397dc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:12.166896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:12.167808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 356/704] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 321 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223fc3c-dff6-4065-98ca-ae86070e56a4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223fc3c-dff6-4065-98ca-ae86070e56a4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.170162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6223fc3c-dff6-4065-98ca-ae86070e56a4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.174881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 357/705] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:12.174881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223fc3c-dff6-4065-98ca-ae86070e56a4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.179459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.179636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.179871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.180142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.189288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.190037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:12.191239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1583d723-f5a4-4325-b984-967c0011cd14 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:12.211537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:12.212919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 349/706] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.214665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.215288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.233579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:12.235138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:12.310774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6223fc3c-dff6-4065-98ca-ae86070e56a4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.317890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 350/707] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:12.317890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.354062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.354737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.355028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.355299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.356035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:12.356348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.357768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af019b92-8ee5-471b-aeb9-34887c715826 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.371626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 358/708] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:12.373168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71797292-5c41-482e-af92-1f6c670ec477 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.373535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71797292-5c41-482e-af92-1f6c670ec477 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.373893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71797292-5c41-482e-af92-1f6c670ec477 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.375343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 359/709] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:12.376641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71797292-5c41-482e-af92-1f6c670ec477 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.405559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:12.435859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f76adc-08a3-4b6f-9210-927efa3ce405 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.453905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:12.473978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.475196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.475712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.476204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.477672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:12.478171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.480249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.498931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 351/710] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:12.499147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71797292-5c41-482e-af92-1f6c670ec477 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.499248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.534707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.535523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.535808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.536083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.536780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.538152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-8aadeed3-ed12-49c1-8ec1-739e6f7bd63e None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:12.552852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:12.553861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:12.554741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 360/711] 10.222.0.70 () {56 vars in 1129 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:12.555602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.555602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.557461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.580238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:12.581472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:12.623382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c1ad-b3dd-4a27-9a66-948925a69013 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.656796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 361/712] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:12.658369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.661350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.662284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.662775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.663253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.664155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.665919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.677596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:12.724142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:12.745449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 352/713] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:10:12 2022] POST /identity/v3/users => generated 254 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:12.746361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.780362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ee92eb-0275-4457-89a6-093a75056899 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.833247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.834014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.834372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.834743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.835850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=9cba93afd3344df298ba353d9994ac37) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:12.836109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.837722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.845614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 362/714] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/role_assignments?user.id=9cba93afd3344df298ba353d9994ac37&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:12.846603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:12.872376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c39cc0-004d-4c3d-9c02-3d81b2523076 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:12.913437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:12.914532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:12.915019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:12.915500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:12.916459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:12.917051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:12.918736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:12.925233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 353/715] 10.222.0.70 () {56 vars in 1124 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:12.934217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:13.031191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:13.058286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7272e3-251e-4a1d-8f24-f1c417a0dd8d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:13.094788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:13.099875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a8b58ef-edd7-4989-8640-162178910e27 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:13.108922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:13.110087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:13.112334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:13.112894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] PATH_INFO: `/v3/services/network` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:13.113901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:13.115678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:13.130400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:13.131506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:13.132377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:13.135450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 363/716] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/services/network => generated 88 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:13.142245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:13.153470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:13.154030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:13.154318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:13.154596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:13.155326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:13.155635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:13.157106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:13.166808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 354/717] 10.222.0.70 () {56 vars in 1124 bytes} [Mon May 23 13:10:12 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:13.266367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9718228b-2a89-4298-9a46-41933aea3ae3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:13.308707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:13.309323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:13.309610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:13.309890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:13.310623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] RBAC: Adding query filter params (name=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:13.310933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:13.312334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc56318-4a91-482a-9032-b27aa943c31b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:13.318731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 364/718] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:13 2022] GET /identity/v3/services?name=network => generated 124 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:13.324215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:13.484658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a765eb93-5a13-483e-9bda-4ff9fa92720c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:13.504353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:13.523001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:13.523702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:13.524016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:13.524308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:13.525054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] RBAC: Adding query filter params (type=network) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:13.525462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:13.527093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c4552b4-df1b-4b12-b0e4-2adca35d511b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:13.532278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 355/719] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:13 2022] GET /identity/v3/services?type=network => generated 124 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:14.520246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b14ddb-f573-42ec-b5f2-6464a97e22ef None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:14.520785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b14ddb-f573-42ec-b5f2-6464a97e22ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:14.520785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b14ddb-f573-42ec-b5f2-6464a97e22ef None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:14.522384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 365/720] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:14.528231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:14.528682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:14.528863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:14.546297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:14.547469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:14.616716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 356/721] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:14.651571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b14ddb-f573-42ec-b5f2-6464a97e22ef None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:14.747045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:14.808465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b14ddb-f573-42ec-b5f2-6464a97e22ef None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:14.857299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:14.858103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:14.858411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:14.858652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:14.879640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:14.887296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:14.888285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:14.889218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:14.892906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 366/722] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:14.895022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:15.059299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da92849e-543d-4f52-a83e-9c0eb89262d0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:15.079456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:15.105013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:15.114120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.114120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.114120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.114120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:15.114120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:15.114120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:15.120309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 357/723] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:15.126127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:15.299593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7911c91-70fb-4563-b850-f909a3f589f0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:15.314228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:15.346939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:15.348039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.348324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.348618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.365382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:15.366838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-12eb5379-fe12-4579-8bf4-e8fb69c724a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:15.387540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:15.388513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:15.390125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:15.390125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:15.390125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:15.390125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:15.390125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 367/724] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:15.395063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:15.396469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee7cf2-69c6-44c7-850e-1b1f6f5bb9ac None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.396778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee7cf2-69c6-44c7-850e-1b1f6f5bb9ac None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.397053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dee7cf2-69c6-44c7-850e-1b1f6f5bb9ac None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.398647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 368/725] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:15.404218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.404744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.405190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.434744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:15.436380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:15.560453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 369/726] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 23 13:10:15.572855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7245ca-7972-446c-9278-0446916e57a3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:15.597515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:15.642867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:15.644643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.645776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.646408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.648337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:15.649046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:15.651504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:15.672387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 358/727] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:15.678984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:15.747022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:15.763369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:15.770187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd793c-5304-4d14-b6a7-dba262af7636 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:15.809295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b489fee9-686e-4df1-8805-1538df866ad7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:15.810488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489fee9-686e-4df1-8805-1538df866ad7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.810940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489fee9-686e-4df1-8805-1538df866ad7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.811222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b489fee9-686e-4df1-8805-1538df866ad7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.825152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64dc966-3e60-46c9-86b9-863e658c996e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:15.825840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b489fee9-686e-4df1-8805-1538df866ad7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b489fee9-686e-4df1-8805-1538df866ad7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:15.834297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:15.835245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:15.836072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:15.839115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 370/728] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:15.840753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370f1f0-9e4b-4d98-a88d-6aa0d855e5ae None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.841294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370f1f0-9e4b-4d98-a88d-6aa0d855e5ae None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.841725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d370f1f0-9e4b-4d98-a88d-6aa0d855e5ae None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.847736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 371/729] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:15.848923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370f1f0-9e4b-4d98-a88d-6aa0d855e5ae None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:15.863106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:15.864074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.864487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.864888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.873204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:15.875070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-83cb43a6-a6f5-4d1d-8457-0172728b9c95 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:15.891832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:15.893115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:15.894181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:15.895157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 359/730] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:15 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:15.896470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:15.896781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:15.897057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:15.915605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:15.916933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:15.965876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d370f1f0-9e4b-4d98-a88d-6aa0d855e5ae None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:15.996735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 360/731] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:15.997850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.011443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.012458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.013015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.013514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.014462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:16.014966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.016786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.029050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 372/732] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:16.030288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.036906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.053077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.125074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bceae7-a1ef-4030-9890-5e620306581e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.146803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0beeca9c-0e40-4cbd-9965-4de6008e8477 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.164383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.165460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.165907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.166507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.167508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:16.167977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.169825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.186801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.187620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.187898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.188246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.203575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 361/733] 10.222.0.70 () {56 vars in 1135 bytes} [Mon May 23 13:10:15 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:16.204887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:16.205072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:16.215107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:16.216095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:16.216962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:16.223251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 373/734] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:16.223251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.306838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.313787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.324818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3441f2b3-7875-41a6-8b05-85f1ad1ca5a8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.338922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157a25b2-2f8c-4d24-966c-2851987cfaca None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.361286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.362513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.362793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.363071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.374926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.376108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4222bc5-408f-41ee-b3c8-b1ce3b223abb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.376657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4222bc5-408f-41ee-b3c8-b1ce3b223abb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.376834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.377115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4222bc5-408f-41ee-b3c8-b1ce3b223abb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.377454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4222bc5-408f-41ee-b3c8-b1ce3b223abb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:16.399988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:16.401466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:16.403169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:16.403169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:16.403169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:16.403169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:16.404707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 362/735] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:16.406188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.452698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4222bc5-408f-41ee-b3c8-b1ce3b223abb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.454761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4222bc5-408f-41ee-b3c8-b1ce3b223abb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.517662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177ab4-f433-45ae-a94e-f30bdadc5a79 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.540622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.557957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.558988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.559277 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.559555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.560365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:16.560714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.562092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a39dbb9-1a75-495c-b0b8-b29ab79abefc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.563802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 374/736] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 23 13:10:16.564240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.565146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de52db0-39b0-46ae-9282-7dd96e6cc02f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.565442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de52db0-39b0-46ae-9282-7dd96e6cc02f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.565712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de52db0-39b0-46ae-9282-7dd96e6cc02f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.566975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 375/737] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:16.567668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de52db0-39b0-46ae-9282-7dd96e6cc02f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.572508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 363/738] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:16.574180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef4429-9133-453d-a1b0-0d862c736337 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.574741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef4429-9133-453d-a1b0-0d862c736337 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.574741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef4429-9133-453d-a1b0-0d862c736337 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.576107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 364/739] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:16.577433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef4429-9133-453d-a1b0-0d862c736337 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.672030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de52db0-39b0-46ae-9282-7dd96e6cc02f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.680161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef4429-9133-453d-a1b0-0d862c736337 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.706504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.707167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.707460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.707740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.708467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:16.708803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.710176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900f930b-7926-4f08-af53-fb272456f985 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.713355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.714150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.714434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.714711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.722002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 376/740] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:16.723331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808a6a-157d-424d-94cc-109616e5aa6a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.723699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808a6a-157d-424d-94cc-109616e5aa6a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.723983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25808a6a-157d-424d-94cc-109616e5aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.726498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.727986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.742344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-25808a6a-157d-424d-94cc-109616e5aa6a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:16.742344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-25808a6a-157d-424d-94cc-109616e5aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:16.745757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:16.746628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:16.747529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:16.747529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:16.747529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:16.747529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:16.747529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 365/741] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:16.747801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.773164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.798268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:16.811674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 377/742] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:16.813020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.813309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.813584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.838740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:16.839931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:16.851596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9255da40-2338-42a7-b7d8-92df115cf1d8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:16.885415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:16.886207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:16.886487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:16.886784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:16.897668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:16.899117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:16.909179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 378/743] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 23 13:10:16.913277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:16.916661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:16.917557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:16.918452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:16.918452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:16.918452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:16.918452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:16.920707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 366/744] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:16.921677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.026846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.027606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.030457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b33c8c-51cb-4a11-b767-d4ca2b0e8361 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.031048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24351d32-e1bb-4c95-b297-202d4957e8e0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.064957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.065618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.065899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.066172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.066889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:17.067197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.067418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.068215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.068531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.068683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.068807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.076410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.077868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.080401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 367/745] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:17.081227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:17.098694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:17.099792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 379/746] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:10:16 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:17.101316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.186036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f8353-4c33-4686-a9a3-d836e03fb346 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.202581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-788eb435-3ca2-4347-893c-9569e8c8dbac None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.220323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.221090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.221374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.221647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.222360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.223753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.237247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.243152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.243152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.243152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.243152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:17.243152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.243152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.253126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 380/747] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:17.253912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.271931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.305790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 368/748] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:17 2022] POST /identity/v3/services => generated 251 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:17.306559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.345960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.359094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e92725-3638-4aa6-968d-7c8323f3ceb9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.410767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.411585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.411874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.412170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.412947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.414363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.435551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193be140-f03c-417f-87bd-ecd9429a8009 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.472443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.473288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.473575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.473853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.481418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.482897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.489335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 381/749] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:17 2022] POST /identity/v3/services => generated 240 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:17.490121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:17.508258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:17.509371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:17.510289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 369/750] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:17.510818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.602146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.623274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.644726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0301fd9-213a-4415-8c8a-27fdbd03ce97 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.647007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a524e481-07ad-4733-9615-4d0c61e815be None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.695905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.697351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.697351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.697351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.698112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:17.698435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.699195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.699305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.699523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.699795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.701434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.708505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.710455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.725589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 382/751] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:17.726727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:17.736221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:17.737207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:17.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:17.738654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 370/752] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:17.739809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.855543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6a5aa5-3987-4552-ba19-97117403507c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.885970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.893082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e399ebf6-74fd-490d-bdab-19ea27e32b15 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:17.894823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.895682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.896489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.896701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.897682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:17.898169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.899779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.916681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 383/753] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:17.918037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:17.936983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:17.937830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:17.938953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:17.939213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:17.951067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:17.954398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:17.955964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:17.977261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:17.978216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:17.979133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:17.979133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:17.979133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:17.979133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:17.983351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 371/754] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:17.990295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.062732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42a91e4-2f5d-439a-9a10-312e5f3f51af None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:18.102032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2136eb6-3311-45b4-a4c9-0c553474a927 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:18.107501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:18.108198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:18.108548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:18.108922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:18.109937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:18.110318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:18.112557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:18.137161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 384/755] 10.222.0.70 () {56 vars in 1149 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:18.138842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.147957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:18.149536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:18.149988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:18.150276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:18.151383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=e525e14157f94459a6e2ab46ffdeb529) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:18.152191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:18.154147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:18.163522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 372/756] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:17 2022] GET /identity/v3/role_assignments?user.id=e525e14157f94459a6e2ab46ffdeb529&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:18.165287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.238310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:18.292090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:18.349487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e9a2ef-d8c3-472f-b279-376c4b137bd7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:18.378337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ffd7241-5075-4408-9b4e-211824947543 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:18.424710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:18.425452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:18.425721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:18.426000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:18.426741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:18.427050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:18.429077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:18.443736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 385/757] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:18.448234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.449852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:18.450754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:18.451250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:18.451723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:18.561459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c47fe2a-0ed3-4c43-babd-10d3d5b7600d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:18.568820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:18.569968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:18.576598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:18.602752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:18.603663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:18.603836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:18.604105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:18.612977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:18.613910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:18.629485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:18.630405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:18.631287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 386/758] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:18.636676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.736649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:18.750197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b1107d-c0be-4a85-8846-1204f54cdc20 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:18.790858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:18.791576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:18.791872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:18.792153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:18.793019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:18.793338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:18.794791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:18.810011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 387/759] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:18.819597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.852912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 373/760] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:18 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 689 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:10:18.858475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:18.894428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:19.071095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62118bd-93d4-4b1d-a59a-e4555b25d764 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:19.145424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:19.146161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.146448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.146724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.177940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:19.179465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:19.203403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200035d2-f3ca-410b-a3b4-9772e8d42704 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:19.210823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:19.211760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:19.212703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:19.212703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:19.212703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:19.212703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:19.213399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 374/761] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:19.220552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:19.239549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:19.316302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:19.317630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.318139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.318642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/9cba93afd3344df298ba353d9994ac37/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.322900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=9cba93afd3344df298ba353d9994ac37, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:19.324608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:19.386769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475b533d-987c-47ae-811a-e6dd9fd586eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:19.409309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 388/762] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:18 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/9cba93afd3344df298ba353d9994ac37/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 591 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:10:19.435037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:19.436117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.436658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.437195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.439544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:19.440145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:19.441852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:19.456263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 375/763] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:19.462105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:19.588427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:19.615335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15de7245-18ad-4499-9501-ce057ede53fd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:19.669014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:19.670239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.670808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.671293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.679995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:19.681877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4b3249d8-38dd-4762-898a-ec8c52529d0d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:19.699882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:19.700877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:19.701780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:19.704855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 389/764] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:19.710880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:19.788479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:19.826780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d37164-ca65-45f6-8c97-c74e843bdcd1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:19.846367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834d3f2-b9c2-4946-844f-4dd2e67bb7bb None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.847499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834d3f2-b9c2-4946-844f-4dd2e67bb7bb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.848002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d834d3f2-b9c2-4946-844f-4dd2e67bb7bb None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.850205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 390/765] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:19.857375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.857978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.858485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.897174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:19.900153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:19.901065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:19.901555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:19.902607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:19.903968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:19.904617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:19.906120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:19.909702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:19.943880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 376/766] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:19.946676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.022581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 391/767] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:20.069975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.127343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdb9fb-f224-4605-bd1e-0f0e76919671 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.163874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:20.167250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.167914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.168193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.168466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.169356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=c7d9b65b064a4326ad0d8e6235b46fd2) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:20.169665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.173055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccb49737-1532-431a-a2e3-7efcf9e9310d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.177907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 377/768] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:19 2022] GET /identity/v3/role_assignments?user.id=c7d9b65b064a4326ad0d8e6235b46fd2&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:20.179379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc549-0677-4f5b-889b-2c4dc263a340 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.179735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc549-0677-4f5b-889b-2c4dc263a340 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.180019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5bc549-0677-4f5b-889b-2c4dc263a340 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.185617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 378/769] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:20.194190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.194445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.194726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.218471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:20.220239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:20.226218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:20.274568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d097dd-b605-49a5-955c-e0eb6c067fd3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.356213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.357417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.357918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.358371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.359246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.361261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.372626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 379/770] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:20.383867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:20.384974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:20.385823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:20.394975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 392/771] 10.222.0.70 () {56 vars in 1129 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 325 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:20.401861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.410323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.539636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78913c47-46e0-458d-8a6a-d34c38f36286 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.541855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f185-fd2c-4b7d-82e7-81f54852baf9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.572537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:20.585798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.604712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.604885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.604885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.605032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.605105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.605189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.605189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] PATH_INFO: `/v3/services/network` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.605333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:20.605333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.605465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.605542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.605612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.605686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 380/772] 10.222.0.70 () {56 vars in 1124 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:20.613099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:20.614313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:20.615136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:20.616317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 393/773] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/services/network => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:20.616820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.620734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.625074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:20.734060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15e6ed7-d3ba-4344-a1ba-47c887dd284c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.770321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.771001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.771286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.771558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.772271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] RBAC: Adding query filter params (name=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:20.772601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.775619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.784820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 394/774] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/services?name=network => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:20.787879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:20.838875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec8c78f-4985-4c1c-b951-55b7b99e9ac4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.861011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:20.898824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac8e5d3-9506-4316-ae4e-9838c1dda4d1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:20.904688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.905393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.905681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.905963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.906787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:20.907102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.908757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8487c46e-d2c7-4dba-8f94-f41d5824ca09 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.915217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 381/775] 10.222.0.70 () {56 vars in 1124 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:20.916740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0c7fb-0df6-4216-a2e4-fec91a92ef27 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.917033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0c7fb-0df6-4216-a2e4-fec91a92ef27 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.917312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c0c7fb-0df6-4216-a2e4-fec91a92ef27 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.918992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 382/776] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:20.925191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.925477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.925758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.944407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:20.945639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:20.953438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:20.954158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:20.954442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:20.954843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:20.955575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] RBAC: Adding query filter params (type=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:20.955880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:20.957280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:20.969719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 395/777] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/services?type=network => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:20.969719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.017246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:21.052345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 383/778] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:21.054469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.117346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c66f5f-138e-4aa8-9392-508b174dcdab None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.151959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:21.157596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.158430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.161089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.161089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.161089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] RBAC: Adding query filter params (interface=public, service_id=bf1cc955504041eb97c5739c5e3ff7e4, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:21.161089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:21.162271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:21.171123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 396/779] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:20 2022] GET /identity/v3/endpoints?service_id=bf1cc955504041eb97c5739c5e3ff7e4&interface=public®ion_id=RegionOne => generated 194 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:21.172318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.199856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bb3101-8be2-4060-846a-8a95473a2392 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.239801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.240984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.241496 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.241976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.243139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] RBAC: Adding query filter params (interface=public, service_id=12479988727d4a4d980ed42b0fda7f83, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:21.243706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:21.245436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:21.255223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 384/780] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/endpoints?service_id=12479988727d4a4d980ed42b0fda7f83&interface=public®ion_id=RegionOne => generated 194 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:21.256455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.286501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f92203a-fbd8-40d7-a745-7a2a8fcebed3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.317855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:21.327416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.328972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.329309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.329618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.341536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:21.349434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:21.350387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:21.351249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 397/781] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:21.351937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.390655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390b8ab-15eb-46cb-8419-8c2f7d790ee6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.414638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:21.429498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.430524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.431028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.431511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.432405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:21.434045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:21.440926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 385/782] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/services => generated 772 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:21.442212 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.465634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f65e1f6-a2e2-485e-9310-93e10b99e2c3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.504150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.504928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.505213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.505489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.506199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:21.507704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:21.514135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 398/783] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/services => generated 772 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:21.587049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225ac326-9793-4e7d-b31c-279894f00d91 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.617010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:21.652825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.653450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.653731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.654007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.654798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:21.655107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:21.657124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:21.667583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 386/784] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:21.677918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:21.799243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ceb51e-b1b6-4ee6-8604-f0d4fe291643 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:21.825247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:21.837385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:21.838203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:21.842253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:21.842253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:21.850177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:21.851635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-71cad039-9894-4bb9-86ef-c0bb5fdad4ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:21.874116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:21.876429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:21.878907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:21.878907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:21.878907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:21.878907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:21.878907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 399/785] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:21.881909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.060693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7c36a1-04a9-4a4e-99a1-07b4b7db576c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:22.106448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:22.107145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.107427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.107704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.108441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:22.108780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:22.110743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31176f2f-1c9b-4ac5-a2b7-fd1522da233d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:22.125899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda18eb4-c279-4215-acc1-24fa08399ce8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.126215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda18eb4-c279-4215-acc1-24fa08399ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.126622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda18eb4-c279-4215-acc1-24fa08399ce8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.128284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 400/786] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:22.130171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 387/787] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:22.137559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda18eb4-c279-4215-acc1-24fa08399ce8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.140342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.140713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.140990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.175976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:22.179718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:22.228947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:22.322262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 388/788] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:22.341769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda18eb4-c279-4215-acc1-24fa08399ce8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:22.358771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.416235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:22.417069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.417346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.417619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.436330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:22.440246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:22.461836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:22.462856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:22.463952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 401/789] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 326 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:22.477289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.516850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:22.586146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f65578-82d3-4cc5-8d66-fc7eac6bebf8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:22.601661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd7edaa-4d54-400a-9408-02b75e18c019 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:22.658557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:22.659381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.659669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.659948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.671685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:22.680242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:22.681264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:22.682180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:22.682816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 389/790] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:22.688454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.693419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:22.694837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.695118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.695389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.696241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:22.696544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:22.698845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:22.732950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:22.737632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 402/791] 10.222.0.70 () {56 vars in 1135 bytes} [Mon May 23 13:10:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:22.746599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.856316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:22.858965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6eec221-1113-42a0-a4c2-ab2444d3ebae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:22.867868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a849af-3108-46d0-9747-fa636897368b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:22.895568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:22.896434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.896639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.896929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.897692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:22.898050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:22.899515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:22.908792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 390/792] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:22.916623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:22.951410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:22.952589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:22.953576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:22.954222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:22.998193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.000953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.021112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:23.048700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-565572c8-ebc0-4b32-a265-5fba63f5dce7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-169605cd-5f0d-4861-ae42-192950a849ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:23.049185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:23.050067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:23.051247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:23.051247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:23.051247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:23.051247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:23.051247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 403/793] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:23.057585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865275b5-457a-4aa9-8372-c2019cb62d8e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.057880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865275b5-457a-4aa9-8372-c2019cb62d8e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.058159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865275b5-457a-4aa9-8372-c2019cb62d8e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.072154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 404/794] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:23 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:23.072154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865275b5-457a-4aa9-8372-c2019cb62d8e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.098053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.098867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.099148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.099416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.114223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.115620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cf003870-4505-4614-a681-9fa1e818be7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:23.139291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:23.140261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:23.142629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:23.142629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:23.142629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:23.142629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:23.144643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 391/795] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:23.146337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.147701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.148917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.169878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:23.171457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:23.197820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865275b5-457a-4aa9-8372-c2019cb62d8e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.236988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.237681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.237975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.238254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.238995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:23.239401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.240809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.252686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 392/796] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:23.253913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.264588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 405/797] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:23.264588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.264588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:23.338011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:23.370317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd813c74-3e57-4ab3-91af-f29ca8b1dca8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.382477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bac6614-94f0-46fe-b0e5-9cb094922a1f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.412403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.413854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.414330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.414721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.425219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.426944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.427427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.428353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.428900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.429409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.430356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:23.430889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.432552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:23.444314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:23.445275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:23.446158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:23.453939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 406/799] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:23.453939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.454531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 393/799] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:23.455703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.531072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:23.563751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d550df08-763e-439d-be6f-ebdbef388afa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.572116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c199f7f8-d016-4e52-9170-7ee63b7fd193 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.590090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:23.601734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.602770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.603292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.603773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.604816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:23.605354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.607042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.611091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.611960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.612247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.612540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.623937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 394/800] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:23.625162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.633161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:23.644382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:23.645497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:23.646514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:23.653920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 407/801] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:23.655146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.768622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b19a75-389f-4285-8e84-129679b7d6f2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.770717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a1067-c2ea-46a4-9d21-a1d2591f49ba None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:23.799010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:23.810647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.811771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.811906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:23.812478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.812848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:23.813133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:23.813442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/e525e14157f94459a6e2ab46ffdeb529/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.813736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:23.815642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=e525e14157f94459a6e2ab46ffdeb529, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.817483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.823774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:23.825529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:23.841290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:23.845475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:23.846427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:23.847514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 395/802] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/users/nova => generated 82 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:23.848413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.910349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 408/803] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:23 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/e525e14157f94459a6e2ab46ffdeb529/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:10:23.912822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:23.914901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.036628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7947cdc-d1e4-4893-abc5-b2065e80e9ec None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.111780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.113433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.113731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.114017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.114797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:24.115108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.116528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.130920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d66d9a-f7c8-4e67-b470-350d77cb0e1c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.132180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 396/804] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:24.138860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.191245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.192286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.192592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.192888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.193752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:24.194065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.195498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7a7264-6c03-4c4f-9299-0fbdf6086f84 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.216152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 409/805] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:10:23 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 305 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:24.217648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c194-f574-43cb-a283-6e3913a5caac None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.218398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c194-f574-43cb-a283-6e3913a5caac None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.219214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16c194-f574-43cb-a283-6e3913a5caac None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.220501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 410/806] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:24.221702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fc1d5-81b0-4f22-abef-b1e255bcc797 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.222089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fc1d5-81b0-4f22-abef-b1e255bcc797 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.222376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01fc1d5-81b0-4f22-abef-b1e255bcc797 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.223871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 411/807] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:24.224604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fc1d5-81b0-4f22-abef-b1e255bcc797 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.260009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2053890d-01cc-4ae0-81dc-4f9c17ca9d43 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.267761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.271749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.295571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.296371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.296710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.297143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.307887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.309693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5576c00a-d9d6-45e7-9f2d-56f23a78303a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:24.328062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:24.329662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:24.330561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:24.330561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:24.330561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:24.330561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:24.330561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 397/808] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:24.331643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10608b0-adfd-4c41-8b94-f0b207f984d7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.331643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10608b0-adfd-4c41-8b94-f0b207f984d7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.331643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10608b0-adfd-4c41-8b94-f0b207f984d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.341219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01fc1d5-81b0-4f22-abef-b1e255bcc797 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.349426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a10608b0-adfd-4c41-8b94-f0b207f984d7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:24.350617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a10608b0-adfd-4c41-8b94-f0b207f984d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:24.376920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.377737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.378370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.378651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.420141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 398/809] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:24.421435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.422382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.422750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.451126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:24.452224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:24.464767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.466348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.506602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.528634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 399/810] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:24 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:24.528634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.549811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.582047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 412/811] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:10:24.582804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.632792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7e0fe6-4e18-4b33-828c-17d7d43ef16e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.668965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.669627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.669909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.670181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.670897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:24.671200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.673128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.684691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 400/812] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:24.685523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.701427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a57f70-65b8-41f1-b50c-bb1bc8cedb37 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.748337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.751558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.752501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.752900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] PATH_INFO: `/v3/services/network` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.753495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.754891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:24.768358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:24.775668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:24.776642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 413/813] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/services/network => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:24.777421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.777421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.784030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:24.810987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d3ea8d1-7a06-4181-857e-f42a11a60ab0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.847567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.848395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.848721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.849000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.849670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.851047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:24.865837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:24.866831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:24.867662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 401/814] 10.222.0.70 () {56 vars in 1129 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:24.868489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.896337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41875aab-ebad-4bf1-b32f-d7ae355d54d0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:24.934258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:24.935060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:24.935345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:24.935619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:24.947680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:24.949392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:24.967951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:24.969105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:24.970401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:24.970401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:24.970401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:24.970401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:24.973875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 414/815] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:24.974744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:24.981236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6751434-fc57-45e2-a6ce-a9c50ebb024c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.018151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.019116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.019401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.019672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.029545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.031989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.033853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:25.051328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:25.053367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:25.054487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 402/816] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:25.055360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.058663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.083896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928ea1b3-1ac6-4620-bfa0-0820ac2afb40 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.121447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.122130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.122414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.122689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.123394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] RBAC: Adding query filter params (name=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:25.123701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.125812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.138536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 415/817] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:24 2022] GET /identity/v3/services?name=network => generated 124 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:25.139272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.193201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2c6d24-6fac-417d-b0b3-d2db0a508854 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.232760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.233346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.233626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.233900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.235447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:25.235754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.237170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.242128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 403/818] 10.222.0.70 () {56 vars in 1124 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:25.243230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.264165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc2c49c-a67e-4625-a1da-08a9272b1d34 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.285563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.301012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.301697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.301983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.302264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.303020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:25.303304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.304696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.315543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.320857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 416/819] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:25.320857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.355292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95271bbd-0659-4687-a402-5788cee295a1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.390936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.391610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.391893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.392173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.393045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:25.393353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.394793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.414204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 404/820] 10.222.0.70 () {56 vars in 1149 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:25.419356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.424862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6259ff-018f-42bf-8b07-775285ef444b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.462682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.463360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.463646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.463925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.464663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] RBAC: Adding query filter params (type=network) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:25.464974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.466403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.476276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 417/821] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:25.476276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.520657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.534209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b62a5d-3816-4a4b-9e20-443d494e753d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.558846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.571438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.572131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.572392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.572690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.573486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:25.573793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.575153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.576910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231be641-45ce-4e75-970c-a4db3ab68683 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.584694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 405/822] 10.222.0.70 () {56 vars in 1124 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:25.584694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.613649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.614456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.614748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.615026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.622064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.623522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:25.637489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:25.638402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:25.639291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:25.643535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 418/823] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:25.644391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.693918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0a3877-13ff-4fbd-aa4e-cf3a048d24a8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.729339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.730149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.730435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.730712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.742517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.743984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.753314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347efe2c-4c3f-4ac8-af8e-1ccebed2e43a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.756602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:25.762580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:25.768401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:25.769321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:25.769321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:25.769321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:25.769321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:25.769321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 406/824] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:25.769321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.796182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.797337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.797546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.797822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.798574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.799940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:25.814807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:25.871532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86940069-33e4-4325-aa2e-ac6390f7b1eb None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:25.899451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 419/825] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:25 2022] POST /identity/v3/endpoints => generated 327 bytes in 255 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:25.900990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:25.924744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:25.925808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:25.926240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:25.926608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:25.927590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:25.929261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:26.004464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 407/826] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:26.005566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.070868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4039d66-725c-4977-81a3-d6625288763f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:26.164098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:26.164825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.165107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.165414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.166960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:26.167272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:26.169105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:26.203366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 420/827] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:26.213788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.237164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:26.252759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28f91f0-2bb0-40f5-bea4-5d78ab01f770 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:26.254937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:26.304903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:26.305984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.306722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.307314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.308677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:26.309155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:26.311193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:26.330430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 408/828] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 325 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:26.341857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.386816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab93a22-9507-4bbc-8ed2-8dbed8d6af73 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:26.426062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:26.426822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.427167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.427509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.428417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=9cba93afd3344df298ba353d9994ac37) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:26.428843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:26.430331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2237165e-92ed-4678-afcb-a627fd9cd7b4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:26.445027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 421/829] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:26 2022] GET /identity/v3/role_assignments?user.id=9cba93afd3344df298ba353d9994ac37&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:26.446735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e02912-a420-45b6-ad60-b36bb5dacea5 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.447096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e02912-a420-45b6-ad60-b36bb5dacea5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.447484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e02912-a420-45b6-ad60-b36bb5dacea5 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.449235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 422/830] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:26.455241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.455598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.455951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.503674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb1b499-2b96-4ef6-8ffe-276979fb13ef None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:26.505995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:26.508469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:26.562073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:26.563533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.563923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.564207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.581342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:26.584959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:26.616587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:26.617658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:26.618547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 409/831] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:26.624641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.650016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:26.666863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 423/832] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:26 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:26.694154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.710840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:26.766923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b66ab81-a1ea-4bc6-b838-86ac75b20e4b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:26.810831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:26.811866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.812291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.813374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.814572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:26.815158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:26.817387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:26.818697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fde18a-74c2-448c-872f-4a43aa486eda None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:26.838778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 410/833] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:26.844481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.870243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:26.871398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:26.871754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:26.872500 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:26.888359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:26.899287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:26.900691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:26.901730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:26.904613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 424/834] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:26.910515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:26.932444 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:26.974122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8784e1fa-cbcf-4a3f-ae99-8f4472fef852 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.010517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.011414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.011694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.011974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/c7d9b65b064a4326ad0d8e6235b46fd2/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:27.012338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:27.028737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=c7d9b65b064a4326ad0d8e6235b46fd2, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:27.030189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:27.051184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abab429-fd63-4a89-aea2-a023935d3434 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.087254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.087929 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.088214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.088495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:27.089329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:27.089588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:27.090968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:27.100247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 425/835] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:27.105718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:27.115074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 411/836] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:26 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/c7d9b65b064a4326ad0d8e6235b46fd2/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:10:27.232380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4529400-cabd-4516-b9a6-d1c71a636dc5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.267582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:27.273441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.274266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.274548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.275080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:27.289165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:27.290644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:27.311593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:27.312725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:27.314078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:27.314078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:27.314078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:27.314078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:27.314078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 426/837] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:27.319110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:27.440681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5f3baee-95ef-4d76-9f53-fd1154397313 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.465279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:27.478342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.479016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.481285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.481285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:27.481285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:27.481285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:27.482123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:27.494364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 412/838] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:27.499920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:27.606016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd88d1b-3689-4367-8d8a-10f882faad76 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.642519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.643699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.643983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.644321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:27.652832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:27.654277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:27.669544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:27.670541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:27.671584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 427/839] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:27.676986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:27.786191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b59add7-f0ea-4888-8c2e-a0943aeaae0a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.821206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.821893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.822175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.822500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:27.823414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:27.823745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:27.825225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:27.844970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 413/840] 10.222.0.70 () {56 vars in 1135 bytes} [Mon May 23 13:10:27 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:27.850828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:27.900980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:27.957459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db77a4b2-5905-4193-b1bd-8b2f6cf4fe87 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:27.994999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:27.995668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:27.996194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:27.996472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.068546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.069943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.117206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:28.179850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 428/841] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:27 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:10:28.185476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.240971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:28.292145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cfaf49-aaf5-4127-aaf0-9f7d24e7aa5e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:28.331715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:28.332495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.332811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.333087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.344760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.346233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-44109652-7f26-43e9-ba0f-d355a7d48b4d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:28.364989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:28.365927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:28.366843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:28.366843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:28.366843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:28.366843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:28.366843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 414/842] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:28.372129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.477707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d902f3d-3ed9-449b-b3d9-3fac01a91e8d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:28.482751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345287f0-cac8-4416-bc68-6ede62502486 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.483133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345287f0-cac8-4416-bc68-6ede62502486 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.483478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-345287f0-cac8-4416-bc68-6ede62502486 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.489690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 415/843] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:28.491365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.491723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.492073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.509505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:28.510775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:28.517527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:28.518288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.518571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.518848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.519575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:28.519882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.521738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.538653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 429/844] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:28.544357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.549618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:28.592778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 416/845] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:28.611516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.636107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:28.657085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022d03cf-3599-4ab9-b0c4-14a9516be5f2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:28.695608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:28.696555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.696869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.697145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.705400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.706919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.720889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15813ed-ee4e-4b43-8dda-3a8c9dcf0aa0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:28.724824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:28.725766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:28.727214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:28.730762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 430/846] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:28.731775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.763861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:28.765153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.765673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.766144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.767005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.768626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:28.782130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:28.783245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:28.784068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:28.791830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 417/847] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:28.793563 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.807424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:28.855320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec81ecc-acad-4695-b6d0-2223cb1ce9d3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:28.891717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:28.897924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:28.898612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.899042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.899447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.900401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:28.901072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.909045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45490eca-27d4-4bcd-bb52-a3b63dee6794 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.911732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b25be7d-8301-4154-9690-1d6d419c2172 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:28.916641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 431/848] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:28.918019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5e31-a6d0-4b42-8144-1e35c9977722 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.918315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5e31-a6d0-4b42-8144-1e35c9977722 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.918598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7c5e31-a6d0-4b42-8144-1e35c9977722 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.920230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 432/849] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:28.923097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5e31-a6d0-4b42-8144-1e35c9977722 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:28.956839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:28.957487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.958193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.958488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.959369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:28.959683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:28.961101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37d1c484-4330-46ef-aeea-ef9d6de7b9a4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:28.971129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 418/850] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:28.971129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:28.971129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:28.971129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:28.985915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:28.987121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:29.067820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7c5e31-a6d0-4b42-8144-1e35c9977722 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.092693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 419/851] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:29.092693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:29.113806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:29.133224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.134013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.134296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.134570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.135415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=e525e14157f94459a6e2ab46ffdeb529) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:29.135729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:29.137316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:29.148608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 433/852] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:28 2022] GET /identity/v3/role_assignments?user.id=e525e14157f94459a6e2ab46ffdeb529&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:29.148608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:29.223809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:29.255261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec5b7a4-c0d5-424b-b6f9-65c6cf40b632 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.308802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.309365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab057de-91f1-47d4-85ea-c115142a42fd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.310253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.310791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.311359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.313102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:29.313804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:29.315471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:29.329075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 420/853] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:29 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:29.354109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.354986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.355273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.355550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.356247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:29.357803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b778069d-a3a0-4864-b836-0d762df70f4b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:29.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:29.373791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:29.375369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 434/854] 10.222.0.70 () {56 vars in 1132 bytes} [Mon May 23 13:10:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:29.382352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:29.493231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac585cd-57b6-4a2e-b12b-b0282d35b735 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.493568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac585cd-57b6-4a2e-b12b-b0282d35b735 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.493830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac585cd-57b6-4a2e-b12b-b0282d35b735 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.495533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 435/855] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:29.502095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.502439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.502660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.523805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7378a20b-2e43-47c3-9414-9f0333bb3805 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.525006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:29.526245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:29.570813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.571966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.572532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.573111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.574001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:29.574597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:29.576334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:29.583299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 421/856] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:29.589258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:29.591640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:29.601778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:29.620710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 436/857] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:29.639255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:29.704765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64f30cb-da21-4559-bd95-f0de555457bd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.745390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.746335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.746623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.746895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.747616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:29.747920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:29.749417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:29.750243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c51e7-0513-4d7b-840e-eded080a2807 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.755768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 422/858] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:29 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:29.792416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.793286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.793572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.793849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.807252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:29.815893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:29.818236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:10:29.820258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:29.824733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 437/859] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:10:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:29.825046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:29.914739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:29.949312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15eb8f0-f8f6-496f-9575-34ec104959c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:29.991486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:29.992197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:29.992671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:29.993143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:29.994170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:29.994453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:29.995859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:30.007602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 423/860] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:10:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:30.013625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:30.065467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:30.129674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816d73f8-e96b-4287-8647-1b806e5aca0c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:30.174299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56473645-0003-4f46-8621-553878b960c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:30.175033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56473645-0003-4f46-8621-553878b960c2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:30.175314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56473645-0003-4f46-8621-553878b960c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:30.175589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56473645-0003-4f46-8621-553878b960c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:30.190381 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56473645-0003-4f46-8621-553878b960c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:30.196591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56473645-0003-4f46-8621-553878b960c2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-56473645-0003-4f46-8621-553878b960c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:30.215864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:30.217040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:30.218765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:30.218765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:30.218765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:30.218765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:30.218765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 438/861] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:30.225740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:30.340589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de5f538-726b-4f6b-8a74-c4d69c033f39 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:30.376915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:30.377595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:30.377880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:30.378155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:30.378887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:30.379199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:30.380597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:30.382231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:30.394443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 424/862] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:30.400464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56473645-0003-4f46-8621-553878b960c2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:30.510504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:30.517458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56473645-0003-4f46-8621-553878b960c2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:30.555613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:30.556447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:30.556758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:30.557081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:30.566091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:30.567530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:30.583604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:30.584643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:30.585594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 439/863] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:10:30 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:30.591665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:30.700981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8dabfc9-4e16-461f-a617-2d763ddcbab3 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:30.739064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:30.739811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:30.740096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:30.740370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:30.741255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:30.741571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:30.743021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:30.766555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 425/864] 10.222.0.70 () {56 vars in 1149 bytes} [Mon May 23 13:10:30 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:30.767979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:30.888137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf31469-6a10-45e3-bb88-ead4a75d57ca None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:30.929378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:30.930038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:30.930320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:30.930597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:30.957098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:31.004031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:31.005664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:31.131668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 440/865] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:30 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:10:31.137117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:31.199785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:31.254103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8247f3-01eb-44e7-a55b-68ab51411cf9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:31.292606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:31.293430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.293715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.293989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.305864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:31.307330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:31.327909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:31.328918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:31.329836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:31.329836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:31.329836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:31.329836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:31.334029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 426/866] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:31.337721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:31.398162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:31.450727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf49d4d-f966-4cc0-94c0-e22f4955c3e0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:31.495510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:31.496192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.496520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.496825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.497558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:31.497897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:31.499346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12731da-f849-46ba-b33b-04ba9c783822 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:31.516703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 441/867] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:31.519039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:31.642248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d558daaa-7e7a-4a8b-a431-65d88a5a3ac8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:31.644829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:31.662165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e74e9-2f40-4413-9562-de4f8516a5cc None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.662484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e74e9-2f40-4413-9562-de4f8516a5cc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.662766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3e74e9-2f40-4413-9562-de4f8516a5cc None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.664432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 442/868] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:31.671212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb407a89-8d2c-475a-9075-f1775451efbb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.671557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb407a89-8d2c-475a-9075-f1775451efbb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.671809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb407a89-8d2c-475a-9075-f1775451efbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.680403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:31.681308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.681594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.681874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.690017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:31.691235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cb407a89-8d2c-475a-9075-f1775451efbb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:31.691507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:31.692873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cb407a89-8d2c-475a-9075-f1775451efbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:31.707704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:31.708705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:31.709567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:31.713785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 427/869] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:31.715202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:31.791338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 443/870] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:31.798781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a641-29b2-435c-b009-0fdbd014b7ef None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.799239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a641-29b2-435c-b009-0fdbd014b7ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.799695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1505a641-29b2-435c-b009-0fdbd014b7ef None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.804694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 444/871] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:31 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:31.812859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.813363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.814217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.832585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:31.833807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:31.834986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8141dfa-02b0-49f1-99e4-5dcf22c247da None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:31.874112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:31.875557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:31.875845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:31.876124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:31.876976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:31.877289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:31.878744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:31.892415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:31.896865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 428/872] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:31.897798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:31.929635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 445/873] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:31 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:31.930887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:31.938298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:32.050493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3f1b2-4992-4dfe-b15f-3338a139dbc1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:32.069290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f11c2d3-cee9-4805-a91d-f0139306c3c0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:32.090055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:32.090881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.091166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.091440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.092102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:32.093684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:32.107997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:32.109650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:32.110497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:32.112077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 429/874] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:31 2022] GET /identity/v3/services/object-store => generated 93 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:32.113049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:32.121614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:32.122294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.122570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.122847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.123704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=c7d9b65b064a4326ad0d8e6235b46fd2) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:32.124017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:32.126314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:32.137460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 446/875] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:31 2022] GET /identity/v3/role_assignments?user.id=c7d9b65b064a4326ad0d8e6235b46fd2&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:32.138422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:32.218412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:32.246405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:32.260068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec3e1a4-3661-4c5a-8d80-637c41417c4c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:32.289819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29790742-79bb-4381-90f9-7d52c954c119 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:32.299470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e4cb04-7c16-415e-8b93-6e46c82bfd8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:32.300161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4cb04-7c16-415e-8b93-6e46c82bfd8f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.300448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4cb04-7c16-415e-8b93-6e46c82bfd8f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.300758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e4cb04-7c16-415e-8b93-6e46c82bfd8f None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.301515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4cb04-7c16-415e-8b93-6e46c82bfd8f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:32.302949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e4cb04-7c16-415e-8b93-6e46c82bfd8f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:32.342309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:32.343501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.343794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.344067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.344822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:32.345136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:32.346533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7761cd8-bbd2-4dca-8b6d-56b273fee527 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:32.351875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 447/876] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:32.353188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de365f-d839-4f34-b7c7-8ef9de60e8f8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.357959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de365f-d839-4f34-b7c7-8ef9de60e8f8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.357959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9de365f-d839-4f34-b7c7-8ef9de60e8f8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.357959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 448/877] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:32.357959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de365f-d839-4f34-b7c7-8ef9de60e8f8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:32.390710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 430/878] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:32 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:32.392011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.392221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.392503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.414162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:32.415454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:32.537071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9de365f-d839-4f34-b7c7-8ef9de60e8f8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:32.541046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:32.547157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 431/879] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:32.569789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:32.587384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:32.588614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.588908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.589490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.590244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:32.590558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:32.593496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76cbdb8b-1a0f-472c-aba8-e9feb060f446 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:32.600627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 449/880] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:32.680215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:32.709983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee739ccd-ec58-48a9-8bf1-c1b39309440d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:32.750313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8388ebf-fb33-4980-a646-afd972a58bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:32.751010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8388ebf-fb33-4980-a646-afd972a58bca None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:32.751297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8388ebf-fb33-4980-a646-afd972a58bca None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:32.751573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8388ebf-fb33-4980-a646-afd972a58bca None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:32.752324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8388ebf-fb33-4980-a646-afd972a58bca None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:32.753913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8388ebf-fb33-4980-a646-afd972a58bca None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:32.895107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 432/881] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:32 2022] POST /identity/v3/services => generated 269 bytes in 326 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:10:34.579223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59754fbd-fd11-4a5f-8098-8f4f7a94c75a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.579731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59754fbd-fd11-4a5f-8098-8f4f7a94c75a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.579731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59754fbd-fd11-4a5f-8098-8f4f7a94c75a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.581489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 450/882] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:34.587581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861c917-7cce-4875-bcf3-993c62053163 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.588128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861c917-7cce-4875-bcf3-993c62053163 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.588128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861c917-7cce-4875-bcf3-993c62053163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.606683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8861c917-7cce-4875-bcf3-993c62053163 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:34.607874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8861c917-7cce-4875-bcf3-993c62053163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:34.693859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 433/883] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:34.716833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59754fbd-fd11-4a5f-8098-8f4f7a94c75a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:34.776847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1aa5-a9c6-49ed-b7f6-b04d889c36e7 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.777494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1aa5-a9c6-49ed-b7f6-b04d889c36e7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.777776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce1aa5-a9c6-49ed-b7f6-b04d889c36e7 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.779843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 434/884] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:34.786406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a6f42-42c4-4de8-b197-b0351aced7e1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.787057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a6f42-42c4-4de8-b197-b0351aced7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.787339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160a6f42-42c4-4de8-b197-b0351aced7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.810153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-160a6f42-42c4-4de8-b197-b0351aced7e1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:34.812336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-160a6f42-42c4-4de8-b197-b0351aced7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:34.832362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59754fbd-fd11-4a5f-8098-8f4f7a94c75a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:34.879527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:34.880354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.880664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.880938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.881560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:34.894732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:34.902372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 435/885] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:34.902372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5184-0475-4b2f-8658-4b487fc8b8b9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.902372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5184-0475-4b2f-8658-4b487fc8b8b9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.902372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02bd5184-0475-4b2f-8658-4b487fc8b8b9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.902372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 436/886] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:34.903746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:34.904708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:34.906148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 451/887] 10.222.0.70 () {56 vars in 1120 bytes} [Mon May 23 13:10:34 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:34.908251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:34.908741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:34.909178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:34.913926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:34.930838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:34.932035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:34.945570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.011710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 437/888] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:34 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:35.011710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.036763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1797676-bb49-4e59-bff0-a7acdd000cbe None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.076600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.077814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.078258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.078538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.079289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:35.079595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.081007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.090061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 452/889] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:10:34 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:35.091006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.129331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b55f656-275f-4c6b-b88a-ee367bddfb25 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.150908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.164506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.165338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.165622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.165894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.166550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.167963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:35.180351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:35.181567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:35.182548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:35.188053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 438/890] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:35.188053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.192681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.210495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd757b6f-ff2d-4378-a913-b8ebdcc97db8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.250363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fcffbb-179d-4440-ad06-c7d52ed97ca4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.251043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcffbb-179d-4440-ad06-c7d52ed97ca4 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.251325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcffbb-179d-4440-ad06-c7d52ed97ca4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.251606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fcffbb-179d-4440-ad06-c7d52ed97ca4 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.252329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcffbb-179d-4440-ad06-c7d52ed97ca4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.253728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fcffbb-179d-4440-ad06-c7d52ed97ca4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.293327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f341da97-326d-4c1b-a6cf-8c4c85db5b52 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.326400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.327381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.327679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.327952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.338141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.339587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:35.357114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:35.358169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:35.359712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:35.359712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:35.359712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:35.359712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:35.359712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 439/891] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:35.360615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.367553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 453/892] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:35 2022] POST /identity/v3/services => generated 241 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:35.368826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be6648-b5fe-4701-b2a8-aadad6b04d23 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.369115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be6648-b5fe-4701-b2a8-aadad6b04d23 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.369392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0be6648-b5fe-4701-b2a8-aadad6b04d23 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.370639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 454/893] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:35.371493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be6648-b5fe-4701-b2a8-aadad6b04d23 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.405654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.484474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b48ffd8-37fa-4807-ad5f-805d3dff70ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.508833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0be6648-b5fe-4701-b2a8-aadad6b04d23 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.526477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.527156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.527437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.527712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.528435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:35.528772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.530241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-826d434c-6ade-4fb2-920d-ef5bcb2540db None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.540558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 440/894] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:35.542131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.542428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.542710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.552825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.553534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.554344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.554639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.555391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:35.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.557328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.560368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:35.561932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:35.568901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.577051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 455/895] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:35.577051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.637953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 441/896] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:35.638562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.675615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.687091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63de831-da31-4c81-9f1d-ef2e13846f80 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.732495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.734114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.734400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.734709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.735723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:35.736033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.737512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.742597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 456/897] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:35.743404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.754856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f0ab82-dd49-4379-8dd8-4823e9bef73b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.797726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.798551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.798837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.799109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.806647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.808122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.817022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:35.825363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:35.832169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 442/898] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:35.833141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.860363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0535af-c7f6-4a24-9f0a-cd90df00d109 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.897283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.898022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.898305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.898578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.899497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.900918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:35.915990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:35.917036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:35.917885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 457/899] 10.222.0.70 () {56 vars in 1132 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:35.918808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:35.932005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:35.945743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99f8551-1d13-4a3c-bc26-79b729eec137 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:35.985103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:35.985765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:35.986048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:35.986324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:35.987118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] RBAC: Adding query filter params (interface=public, service_id=da2def4cbbdc446c89a4b9016e6387a3, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:35.987429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:35.988965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:35.994733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 443/900] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/endpoints?service_id=da2def4cbbdc446c89a4b9016e6387a3&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:35.995543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.044236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7b7a3d-7047-4654-ab1a-775f4472d576 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.074966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:36.084397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.085401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.085985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.086309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.087129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:36.087549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.089036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.113703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 458/901] 10.222.0.70 () {56 vars in 1149 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:36.113703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.113985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec314f7-8b13-45c3-974b-c61316c5e3fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.151379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.151972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.152257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.152533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.153276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:36.153633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.155099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.164641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 444/902] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:36.164641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.183796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:36.231235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9858350e-3e4f-47c3-b703-57ae2974899c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.273424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87374b70-e2e0-4be4-a23d-fa17dab97bc1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.278586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.279270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.279559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.279838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.281364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.282780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.289036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 459/903] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/services => generated 1497 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:10:36.289850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.340595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.341603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.341894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.342173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.361627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:36.424503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc6329b-f6c1-455d-82e5-72b0819856f0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.441051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.443530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.471013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:36.477168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.478739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.478739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.478739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.479976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:36.480429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.482086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.497330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 460/904] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:36.498331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.579887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 445/905] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:10:36.587636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.643275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b74108-8ad6-4357-b1bb-7e1ae9e842bf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.664032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:36.687739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.688786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.689084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.689360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.690207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] RBAC: Adding query filter params (interface=public, service_id=12da1d59edf74fcea27fb80a37648aba, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:36.690492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.691882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.698474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 461/906] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/endpoints?service_id=12da1d59edf74fcea27fb80a37648aba&interface=public®ion_id=RegionOne => generated 194 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:36.704089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.732727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30d7a9a3-85e9-4412-9050-dce25768797e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.753622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:36.773493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.774955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.775634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.776202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.789105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.790890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:36.810186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:36.811132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:36.812561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:36.812561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:36.812561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:36.812561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:36.812810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28b8f3-953f-4514-8759-26b63d56f10c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.819756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 446/907] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:36.820967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:36.848758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:36.849432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:36.849713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:36.849984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:36.850674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:36.852027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:36.864532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 462/908] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:10:36.958238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c83c55f9-b6e0-48f1-a84c-25c8afb5becc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:36.998468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:37.000859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:37.001337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:37.001767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:37.002713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:37.003251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:37.004903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:37.019744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 447/909] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:37.025825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:37.081892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:37.144209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ab81be-bb32-42c0-9ec7-8ed6152bef22 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:37.192734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:37.193559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:37.193848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:37.194125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:37.202753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:37.204349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:37.220080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:37.221553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:37.222995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 463/910] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:37.231539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:37.263745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:37.337627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1018c6bd-10d9-4b24-975a-cb62964a240e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:37.374394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:37.375072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:37.375358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:37.375638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:37.376432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:37.376778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:37.378219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f479691-7481-4b79-9ac7-1092d6a454d5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:37.391491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 448/911] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:37.397407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:37.504409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4715758e-557e-4492-82df-0455b4d0af37 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:37.506170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:37.540717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:37.541425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:37.542160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:37.542447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:37.543322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] RBAC: Adding query filter params (role.id=71a1477a2ccb4e32a791c47b59a6ea89, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=c7d9b65b064a4326ad0d8e6235b46fd2) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:37.544045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:37.545682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a597cf-fda6-4644-85fe-08aa6121a234 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:37.552680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 464/912] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:37 2022] GET /identity/v3/role_assignments?user.id=c7d9b65b064a4326ad0d8e6235b46fd2&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=71a1477a2ccb4e32a791c47b59a6ea89 => generated 259 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:37.751886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20b856-7fc8-4b56-bcb9-d2175adbf64d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:37.752388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20b856-7fc8-4b56-bcb9-d2175adbf64d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:37.752462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b20b856-7fc8-4b56-bcb9-d2175adbf64d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:37.753984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 449/913] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:37.759562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:37.759987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:37.760271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:37.777691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:37.778884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:37.848644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 465/914] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:37 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:37.871976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20b856-7fc8-4b56-bcb9-d2175adbf64d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:37.973131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:37.979119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b20b856-7fc8-4b56-bcb9-d2175adbf64d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:38.015283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:38.016026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.016308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.016595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.017643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:38.019026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:38.031814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:38.032867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:38.033899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 450/915] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:37 2022] GET /identity/v3/services/object-store => generated 93 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:38.041277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:38.151900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:38.157858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88ca3b-2924-4de1-89cf-df8eac49a770 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:38.195230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:38.195914 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.196204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.196489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.197246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:38.197563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:38.199806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:38.206189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 466/916] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:38.211997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:38.322014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02437aa2-5d2f-45cb-88a7-fef4a5d02653 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:38.363240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:38.363927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.364216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.364502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.365252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:38.365563 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:38.366947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:38.372455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 451/917] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:38.378576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:38.493497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1647025a-ab34-4459-9062-28d225879f52 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:38.534026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:38.535038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.535506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.535959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.536968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] RBAC: Adding query filter params (interface=public, service_id=8140a5d98dda4c258de8c01530cdbbcb, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:38.537463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:38.539023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220c8c1e-46b3-4bf9-a7b8-85562aaebbe9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:38.544956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 467/918] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:38 2022] GET /identity/v3/endpoints?service_id=8140a5d98dda4c258de8c01530cdbbcb&interface=public®ion_id=RegionOne => generated 194 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:38.550760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:38.572164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:38.669491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f3cb82-cca5-4709-a600-b5b51228e2c2 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:38.688750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de45144-a3a6-4b45-be23-868f42deb585 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.689050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de45144-a3a6-4b45-be23-868f42deb585 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.689451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de45144-a3a6-4b45-be23-868f42deb585 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.692262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 468/919] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:38.696539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.696926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.697279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.715325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:38.716652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:38.718694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:38.719870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:38.721087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:38.721416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:38.722131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:38.723530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:38.730873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 452/920] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:38 2022] GET /identity/v3/services => generated 1497 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:10:38.740436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:38.804794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 469/921] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:38.824403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:38.938715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:38.958038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6230a-5885-44bf-8df6-4a7f6703d035 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.005227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.006042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.006331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.006609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.007273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:39.008683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:39.027091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:39.028131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:39.029006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 453/922] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:38 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:39.036923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:39.142719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d884daaa-cf52-4b30-95cf-f0650235dcf6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.181876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.182557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.182840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.183113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.183827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:39.184132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:39.185545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394ba357-c93a-4cfe-8826-115c47a754f0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:39.196256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 470/923] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:39 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:39.201704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:39.308248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a6879f-72cd-4a7b-b340-a13a1064bca6 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.343892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429dc6f-3733-4a52-b79a-7d16d3af511d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.344183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429dc6f-3733-4a52-b79a-7d16d3af511d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.344464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a429dc6f-3733-4a52-b79a-7d16d3af511d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.347150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 471/924] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:39.353007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.353550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.354038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.356938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.357617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.358448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.358777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.359683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:39.359997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:39.361429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:39.366992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 454/925] 10.222.0.70 () {56 vars in 1114 bytes} [Mon May 23 13:10:39 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:39.373246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:39.377627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:39.379177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:39.392035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:39.395446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:39.461865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 472/926] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:39.483735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:39.491620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2831db1e-f068-4f33-8c12-047dacf8f3c0 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.528623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.530425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.530968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.531485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.532457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:39.534218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:39.596335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970689e3-920f-4ae4-a967-39bb32d71f70 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.631371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.632198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.632493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.632790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.633501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:39.634880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:39.648640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:39.649910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 23 13:10:39.650736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:39.652408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 455/927] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:39 2022] POST /identity/v3/endpoints => generated 346 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:39.655810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 473/928] 10.222.0.70 () {56 vars in 1132 bytes} [Mon May 23 13:10:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:39.656870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:39.675086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:39.775371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2498fab8-8d70-4ead-9219-3dc09bb3d856 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.822895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.823604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.823867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.824144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.824920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:39.825235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:39.826887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eceb9ba-1160-43fe-9c40-5358e07b397a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:39.833353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 474/929] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:39.839274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:39.916957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:39.953169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73a5d20-3292-48db-82bc-434e46ba1b04 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:39.995418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:39.996515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:39.997051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:39.997536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:39.998551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:39.999890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:40.001540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:40.007681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 456/930] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:40.013574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6fdc6-806e-4f31-b308-d5769245fe6d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.014438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:40.014818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6fdc6-806e-4f31-b308-d5769245fe6d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.015335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d6fdc6-806e-4f31-b308-d5769245fe6d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.017502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 475/931] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:40.024851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.026020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.026517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.051333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:40.053209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:40.143601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-032cd332-5ede-4248-8ea8-567077bc4ddc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:40.169266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 476/932] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:40 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:40.190017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:40.192332 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:40.197176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:40.206464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:40.208065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.208529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.209140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.210511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:40.212107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:40.302436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3146ec74-64e7-4081-a33b-7b6161fd3a9b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:40.337654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 457/933] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:40 2022] POST /identity/v3/endpoints => generated 347 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:40.354538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:40.355468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.355755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.356082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.370591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:40.383146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:40.384762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:40.385812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 477/934] 10.222.0.70 () {56 vars in 1120 bytes} [Mon May 23 13:10:40 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:40.393591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:40.518399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5291f8-fca8-4469-8900-cda5a9fad54f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:40.559819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:40.560503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.560822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.561648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.562408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:40.562717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:40.564305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:40.578194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 458/935] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:10:40 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:40.579567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:40.647852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:40.705254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410c808-7d5d-4221-9426-c600702de09b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:40.744161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:40.745059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.745346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.745695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.757557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:40.759033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:40.777729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:40.778649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:40.780219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:40.780219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:40.780219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:40.780219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:40.784164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 478/936] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:40.785573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:40.842500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:40.897265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485f9962-227c-4bc0-aa5c-978418856b0c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:40.936406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:40.937248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:40.937539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:40.937823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:40.938651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:40.938966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:40.940439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0d2722-34d4-44e3-ad75-a222771e5f93 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:40.960735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 459/937] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:40.962289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.103105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea78c3c-ef40-4ed9-9931-870952a032b0 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.115349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:41.131046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb66f60-8dda-4659-933e-6abe548faea9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.131358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb66f60-8dda-4659-933e-6abe548faea9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.131610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb66f60-8dda-4659-933e-6abe548faea9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.133544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 460/938] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:41.145003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.145547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.145959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.149342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.150335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.150768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.151190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.160624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.162325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.166582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:41.167914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:41.179126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:41.180161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:41.181135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:41.184622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 479/939] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:10:40 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:41.194068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.272929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 461/940] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:41.296262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.336949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:41.339154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d2e24e1-83d6-4ad5-9808-ec0dc121e17d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.399144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.400293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.401856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.401856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.403029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:41.403584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.406832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.410237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36acadd-533f-4180-a9dd-1c5ac71c2831 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.434980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 480/941] 10.222.0.70 () {56 vars in 1149 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:41.444028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.447360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.448429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.448940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.449436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.450324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.451845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:41.464263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:41.466641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:41.469100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 462/942] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:41.472218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.499989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:41.577917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c684f-364b-41c9-a5b9-c4dfd650156b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.579677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:41.588006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621ae1f5-1ad8-4a96-bcde-c43a0400cfa8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.615942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.616791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.617169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.617549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.618447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:41.618851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.620337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.630491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 463/943] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:41.630664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.631270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.631376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.631581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.633011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.647153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.649256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:41.672192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:41.673239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:41.674204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:41.674204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:41.674204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:41.674204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:41.674204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 481/944] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:41.682521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.769356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cc53d7-379a-47f8-a807-61ca0465c639 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.793367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:41.805543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d397b0-8d45-4907-9931-3b7ddc02db39 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:41.821355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.822621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.822980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.823333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.824421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:41.824825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.827505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.838760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 464/945] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:41.840264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.851185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:41.851873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:41.852166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:41.852450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:41.853240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:41.853551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:41.855296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:41.872648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 482/946] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:41.877788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:41.882025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:41.969160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3871643f-f8d2-4f2e-adfa-6edcabf9f778 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:42.003550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59504f96-42fe-44a7-8d73-873add0e4e29 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:42.013783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f46de2-b29d-4a22-98f0-5a2af674b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:42.015034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f46de2-b29d-4a22-98f0-5a2af674b439 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.015355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f46de2-b29d-4a22-98f0-5a2af674b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.015982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f46de2-b29d-4a22-98f0-5a2af674b439 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.016778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f46de2-b29d-4a22-98f0-5a2af674b439 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:42.018780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f46de2-b29d-4a22-98f0-5a2af674b439 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:42.043087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:42.043971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.044261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.044538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.051921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:42.053481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:42.056885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:42.070399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:42.077954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:42.079035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 483/947] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:42.081147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:42.107714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 465/948] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 268 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:42.220112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a367b3-aaba-4a2a-a74e-993b65db6d11 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:42.269090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:42.270045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.270577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.271029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.272076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:42.272607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:42.274781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:42.275605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36d66-cfbc-4b45-87cf-8e3c33655993 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.275896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36d66-cfbc-4b45-87cf-8e3c33655993 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.276172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e36d66-cfbc-4b45-87cf-8e3c33655993 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.277852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 466/949] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:42.284151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.284440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.284745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.293513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 484/950] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:42.301797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:42.308125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:42.309385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:42.379339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:42.399571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 467/951] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:42.423944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:42.434678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-756a09f6-b4eb-48c4-82a0-1853a3cff7fa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:42.445836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:42.484443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:42.485233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.485522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.485810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/c7d9b65b064a4326ad0d8e6235b46fd2/roles/71a1477a2ccb4e32a791c47b59a6ea89` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.488314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=c7d9b65b064a4326ad0d8e6235b46fd2, role_id=71a1477a2ccb4e32a791c47b59a6ea89)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:42.489805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:42.543682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1907c71-0b0c-490c-b8b3-014f2d5611cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:42.576097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 485/952] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:10:42 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/c7d9b65b064a4326ad0d8e6235b46fd2/roles/71a1477a2ccb4e32a791c47b59a6ea89 => generated 0 bytes in 275 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:10:42.587413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:42.587413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.587413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.587413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.587897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:42.587897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:42.587897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING keystone.server.flask.common [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 13:10:42.634711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.sql.core [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 13:10:42.634711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 13:10:42.634711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [parameters: {'id': '1d2ac4daa0aa47bea19009018a16e85b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 13:10:42.634711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80395) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fd355947-86ad-4bda-97a4-4f10b29e612e None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d2ac4daa0aa47bea19009018a16e85b.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d2ac4daa0aa47bea19009018a16e85b. May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 13:10:42.654718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 23 13:10:42.655667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return project_ref.to_dict() May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application yield resource May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application next(self.gen) May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.rollback() May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.force_reraise() May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise self.value May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application session.commit() May 23 13:10:42.657066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._prepare_impl() May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.session.flush() May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._flush(objects) May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application compat.raise_( May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application flush_context.execute() May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 13:10:42.658179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rec.execute(self) May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application c = connection._execute_20( May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 13:10:42.659085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result = self._query(query) May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application conn.query(q) May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application result.read() May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 13:10:42.659941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [parameters: {'id': '1d2ac4daa0aa47bea19009018a16e85b', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:42.662439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:42.663276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.Conflict( May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 1d2ac4daa0aa47bea19009018a16e85b. May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:42.664158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 468/953] 10.222.0.70 () {60 vars in 1140 bytes} [Mon May 23 13:10:42 2022] POST /identity/v3/projects => generated 262 bytes in 233 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:42.665833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:42.786367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db18a1f4-d907-4d1e-ad59-6a8a098c424f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:42.804373 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:42.824654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:42.825477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.825767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.826047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.833768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:42.835228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:42.845289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb51a19-1200-4134-a7dc-0d0204b16585 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.845597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb51a19-1200-4134-a7dc-0d0204b16585 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.845840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb51a19-1200-4134-a7dc-0d0204b16585 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.850352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 469/954] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:42 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:42.852066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:42.852982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:42.853870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 486/955] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:42.855760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:42.856044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:42.856315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:42.861850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:42.875931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:42.877550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:42.962680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 470/956] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:42.986436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:42.988908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:42.994835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698ffc9a-53b4-43ea-a4c9-8381408d2011 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:43.045483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:43.046976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:43.047964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:43.048468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:43.049771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:43.050472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:43.052821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:43.079337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 487/957] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:42 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:43.103164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849c710a-3a1a-4769-917a-0c5c5e93c489 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:43.147353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:43.148194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:43.148472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:43.149031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:43.149715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:43.151346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:43.164613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:43.165783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:43.167081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 471/958] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:43.173342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:43.224810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:43.297691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76edf361-d6bb-4e69-8df7-9c5b93c0ff1e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:43.335866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:43.336539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:43.336852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:43.337131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:43.337840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:43.338156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:43.339505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85662be-0624-4035-a0e3-7dc662cfb1a6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:43.345313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 488/959] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:43 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:43.350854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:43.425021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:43.460978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c00d723f-e389-409a-b131-8b3850792c30 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:43.501507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:43.502487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:43.502949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:43.503428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:43.504313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:43.504847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:43.506404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2f26cc-cd3c-4a29-a569-91307272c38d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:43.511971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 472/960] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:44.570298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ced4c-758a-4209-bfed-fc35ec2d13f6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:44.570833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ced4c-758a-4209-bfed-fc35ec2d13f6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:44.570833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72ced4c-758a-4209-bfed-fc35ec2d13f6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:44.572635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 489/961] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:44.578489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:44.578965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:44.579050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:44.596302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:44.597550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:44.672785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 473/962] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:44.691439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ced4c-758a-4209-bfed-fc35ec2d13f6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:44.801526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:44.805445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72ced4c-758a-4209-bfed-fc35ec2d13f6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:44.843144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:44.843878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:44.844160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:44.844433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:44.845138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:44.846510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-bd39c929-0964-4799-8779-1c59b7528f86 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:44.859734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:44.860884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:44.862008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 490/963] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:44.869936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:44.880716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e430-44aa-4996-8a48-22783d83055a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:44.881042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e430-44aa-4996-8a48-22783d83055a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:44.881283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e430-44aa-4996-8a48-22783d83055a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:44.882899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 491/964] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:44.887872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:44.888163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:44.888447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:44.910667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:44.911874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:44.945189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:44.987027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcce78d2-32e5-4c24-b1a3-780906f36fc5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.008776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 492/965] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:45.029133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.030355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.030862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.031415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.032549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:45.033109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.033434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.034974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.045889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 474/966] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:45.047577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.132795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:45.139343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92ffb7f-6398-4926-93cd-643f86faef1c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.162078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7292fd02-eb0d-429f-89b9-20dee3f330f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.176802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.177492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.177779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.178058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.189913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:45.193574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:45.198468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:45.199458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:45.200314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:45.204849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 493/967] 10.222.0.70 () {56 vars in 1120 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:45.207935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.210388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.211311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.211818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.212308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.213310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:45.213843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.215542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.227240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 475/968] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:45.228907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.312599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d231d-0a2b-4ca6-b1b2-0f966541c922 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.346475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f270a7-2c5f-4563-b024-8dc87726b8fa None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.349425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.350224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.351430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.351430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.351619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:45.352029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.353615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.368533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 494/969] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:45.370554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.393855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:45.397999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.399256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.399755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.400257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.401435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] RBAC: Adding query filter params (interface=admin, service_id=8140a5d98dda4c258de8c01530cdbbcb, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:45.406046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.406046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.415385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 476/970] 10.222.0.70 () {56 vars in 1249 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/endpoints?service_id=8140a5d98dda4c258de8c01530cdbbcb&interface=admin®ion_id=RegionOne => generated 193 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:45.417133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.480591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:45.482567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a37ec5-0245-4d2a-b584-153a47ca4ba5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.519971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.520898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.521116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.521370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.532497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.534000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.539469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6491835f-75f5-4668-b05b-504b6a554830 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-05cf3c9a-adee-4ad8-badd-c22d05ed2965 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:45.553240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:45.554213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:45.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:45.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:45.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:45.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:45.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 495/971] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:45.555701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238edc2-ed90-4caa-8ae8-c5486a04375a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.556083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238edc2-ed90-4caa-8ae8-c5486a04375a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.556294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e238edc2-ed90-4caa-8ae8-c5486a04375a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.557757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 496/972] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:45.563575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.563748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.564296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.583130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.583695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:45.584059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.584059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.584394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.584498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:45.585063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.586456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.595204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 477/973] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:10:45.596058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.686202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:45.695897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 497/974] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:45.723642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.730363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa1cdb7-220a-45dc-80c6-38238c12072b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.752988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:45.774634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.775330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.775615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.775894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.776680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:45.776995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.778434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.799216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 478/975] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:45.801162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:45.872600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9494d7-861c-4c37-a8c9-bb81825722ef None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.915284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.915970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.916257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.916534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.917272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.918761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.920708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING keystone.server.flask.common [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 23 13:10:45.921494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d60c310-2724-4349-bdea-4289a4fb0954 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:45.957020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:45.957758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:45.958046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:45.958324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:45.965986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:45.968244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:45.971005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.sql.core [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 13:10:45.971005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 13:10:45.971005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [parameters: {'user_id': '7b6c68405d1f4cac8b4db712b1f86c18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 13:10:45.971005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80396) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-92066b7a-79ee-4880-a314-1c99da8f3688 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result = self._query(query) May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application conn.query(q) May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result.read() May 23 13:10:45.978435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 13:10:45.985476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application next(self.gen) May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application yield resource May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application next(self.gen) May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.session.rollback() May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.force_reraise() May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise self.value May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application session.commit() May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 23 13:10:45.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._prepare_impl() May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.session.flush() May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._flush(objects) May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application compat.raise_( May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application flush_context.execute() May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rec.execute(self) May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 23 13:10:45.987452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application _emit_insert_statements( May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result = connection._execute_20( May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ret = self._execute_context( May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 23 13:10:45.988378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.dialect.do_execute( May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result = self._query(query) May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application conn.query(q) May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application result.read() May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 23 13:10:45.992293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application packet.raise_for_error() May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [parameters: {'user_id': '7b6c68405d1f4cac8b4db712b1f86c18', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:45.994578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 23 13:10:45.997608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 498/976] 10.222.0.70 () {60 vars in 1131 bytes} [Mon May 23 13:10:45 2022] POST /identity/v3/users => generated 160 bytes in 257 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db224b99-79c4-4d9c-b404-456eee021639 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db224b99-79c4-4d9c-b404-456eee021639 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.001925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db224b99-79c4-4d9c-b404-456eee021639 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2035e683-5ad8-4a99-a43a-203cb29c9788 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:46.003287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:46.006208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 23 13:10:46.009993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:46.011529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 499/978] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:45 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:46.011691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 479/978] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:46.012007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db224b99-79c4-4d9c-b404-456eee021639 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:46.012168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.012168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.012168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.013415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:46.014922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:46.059994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:46.063444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:46.112591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 480/979] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:46.112591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:46.112928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db224b99-79c4-4d9c-b404-456eee021639 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:46.153552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:46.155050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.155620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.155620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.167647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:46.169280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:46.189919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:46.191006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 500/980] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:45 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:46.196851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:46.236459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c5f84e-6f91-4147-994b-7cc3525dce9d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:46.272443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:46.273242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.273878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.274218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.275145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:46.275516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:46.277070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:46.296087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 481/981] 10.222.0.70 () {56 vars in 1149 bytes} [Mon May 23 13:10:46 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:46.296959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:46.304706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d76299-0065-4155-8b16-8eea965ea623 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:46.325631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:46.326642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:46.345755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:46.346690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.346883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.347230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.348034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:46.349589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:46.404314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971b19a-fe0a-4993-b095-39c031c4de05 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:46.409775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 501/982] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:46 2022] POST /identity/v3/services => generated 242 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:46.410851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:46.457099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:46.457791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.458072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.458346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.459221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:46.459526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:46.461183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:46.503786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 482/983] 10.222.0.70 () {56 vars in 1101 bytes} [Mon May 23 13:10:46 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:46.579199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55475ffd-2e15-4831-9ff9-9d963bb250dd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:46.630609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:46.631300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.631574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.631863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:46.695155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:46.718736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:46.720246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:46.836531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 502/984] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:46 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:10:46.842535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:46.931388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:46.954055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3b880f-6b08-42d5-9adb-81f8a658c05c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:46.992677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:46.993494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:46.993780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:46.994059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:47.004837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:47.006306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:47.025089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:47.026071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:47.026979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:47.026979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:47.026979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:47.026979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:47.026979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 483/985] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:47.033695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:47.142573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:47.162843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5fda85-e9f3-417d-870b-67058dd55c35 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:47.201987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:47.202652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:47.202937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:47.203214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:47.203942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:47.204245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:47.205634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:47.222367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 503/986] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:47.223488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:47.324948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fd7966e-2df2-4547-be4e-4e6e7ff53ada None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:47.353808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:47.360319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:47.361209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:47.368058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:47.368058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:47.368436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:47.369908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:47.383869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:47.384827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:47.385730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 484/987] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:47.392134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:47.492826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82b32b-65a3-452a-a753-e44445040e75 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:47.526315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:47.526978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:47.527316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:47.527590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:47.528402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:47.528741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:47.530712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3253cf7d-8c90-45cf-9105-23d6802b78bc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:47.539982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:47.544002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 504/988] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:47.549999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:47.654198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58e501c-461f-4509-be83-7ed0fa5c8d6f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:47.688687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:47.689345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:47.689627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:47.689902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:47.690733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] RBAC: Adding query filter params (role.id=71a1477a2ccb4e32a791c47b59a6ea89, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=c7d9b65b064a4326ad0d8e6235b46fd2) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:47.691043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:47.692491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f274861-b499-4ecb-9d50-6991ca82003e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:47.698773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 485/989] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:47 2022] GET /identity/v3/role_assignments?user.id=c7d9b65b064a4326ad0d8e6235b46fd2&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=71a1477a2ccb4e32a791c47b59a6ea89 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:47.996474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29297-a721-49f9-a5c2-b96aad131632 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:47.997059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29297-a721-49f9-a5c2-b96aad131632 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:47.997059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced29297-a721-49f9-a5c2-b96aad131632 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:47.998891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 505/990] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:48.004910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.005388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.005466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.023407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:48.024704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:48.083532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:48.102528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 486/991] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:48.126826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29297-a721-49f9-a5c2-b96aad131632 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:48.239316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced29297-a721-49f9-a5c2-b96aad131632 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:48.276429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:48.277206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.277489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.277765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.278426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:48.280000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:48.292434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:48.293476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:48.299847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 506/992] 10.222.0.70 () {56 vars in 1126 bytes} [Mon May 23 13:10:48 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:48.302412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:48.409522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76c0390-1f1f-4fda-bc2f-acd8e81063e1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:48.445467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:48.446141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.446429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.446708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.447420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:48.447731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:48.449263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:48.459721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 487/993] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:48 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:48.461257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:48.533307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:48.578108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05f36c5-2e6d-4e47-b847-b8d07e9d7ce7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:48.619127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:48.619807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.620095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.620375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.621125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:48.621438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:48.622830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee72ca1-a6a5-481b-b48e-5466df766c54 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:48.634558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 507/994] 10.222.0.70 () {56 vars in 1122 bytes} [Mon May 23 13:10:48 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:48.634729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:48.685257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:48.739283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6392c2e5-11e2-4c07-9687-8f8991d14be7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:48.768557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0bd8-67b8-4a96-acdf-512f8c0d9d46 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.768880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0bd8-67b8-4a96-acdf-512f8c0d9d46 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.769159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6de0bd8-67b8-4a96-acdf-512f8c0d9d46 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.770667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 508/995] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:48.776028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.776321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.776438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:48.776644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.777061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:48.777351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:48.777639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:48.778401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:48.780306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:48.795258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:48.796892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:48.866809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 509/996] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:48.879356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 488/997] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:48 2022] POST /identity/v3/endpoints => generated 325 bytes in 245 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:10:48.892199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:48.963061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:49.019600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604ce3ef-aed3-4ebf-ad6d-42a410cb9707 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:49.144630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:49.145451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:49.145740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:49.146021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] PATH_INFO: `/v3/services/compute` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:49.146675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:49.148040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:49.161848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:49.162874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:49.163801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:49.165132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 510/998] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:49.169701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:49.284402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d35d90-83e0-4b1b-88df-52f8851494dc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:49.293991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:49.321282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:49.322273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:49.322273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:49.322590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:49.323176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:49.323481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:49.324892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:49.335025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 489/999] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:49 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:49.336701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:49.447458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb1f75-de03-4000-891c-1f55fc07194a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:49.465019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:49.484463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:49.485175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:49.485464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:49.485741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:49.486453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:49.486765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:49.488131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:49.499082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 511/1000] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:49 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:49.500396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:49.607737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2d57ad-4ceb-476c-9d5c-b0ccfb9f3059 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:49.643121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:49.643792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:49.644079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:49.644359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:49.645184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] RBAC: Adding query filter params (interface=public, service_id=1d5b3c6c82264bc3a81230cd54554ba3, region_id=RegionOne) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:49.645508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:49.646891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4896e9d9-3714-45a0-ac20-bc1eb7cad939 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:49.657738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 490/1001] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:49 2022] GET /identity/v3/endpoints?service_id=1d5b3c6c82264bc3a81230cd54554ba3&interface=public®ion_id=RegionOne => generated 194 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:49.658742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:49.766063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b25619c-6f55-46f9-b138-1d6df58519dc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:49.801289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e3a82c-195e-41ad-894f-4719e4349c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:49.801961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3a82c-195e-41ad-894f-4719e4349c9a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:49.802250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3a82c-195e-41ad-894f-4719e4349c9a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:49.802530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e3a82c-195e-41ad-894f-4719e4349c9a None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:49.803217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3a82c-195e-41ad-894f-4719e4349c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:49.804588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9e3a82c-195e-41ad-894f-4719e4349c9a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:49.816882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 512/1002] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:49 2022] GET /identity/v3/services => generated 1727 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:10:50.026320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960d1e-9e5d-49e7-9f95-f99c937f1065 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:50.026522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960d1e-9e5d-49e7-9f95-f99c937f1065 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:50.026811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b960d1e-9e5d-49e7-9f95-f99c937f1065 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:50.028239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 491/1003] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:50.034434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:50.034989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:50.034989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:50.052931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:50.054127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:50.098860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:50.124599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 513/1004] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:50.146529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960d1e-9e5d-49e7-9f95-f99c937f1065 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:50.194908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:50.267661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b960d1e-9e5d-49e7-9f95-f99c937f1065 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:50.302732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:50.303446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:50.303729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:50.304005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] PATH_INFO: `/v3/services/image` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:50.304680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:50.306063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:50.318948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:50.319987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:50.320872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 492/1005] 10.222.0.70 () {56 vars in 1105 bytes} [Mon May 23 13:10:50 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:50.324757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:50.427829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca4ff0-71a6-453f-8425-1fe034733b93 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:50.463069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:50.463737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:50.464018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:50.464291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:50.465023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] RBAC: Adding query filter params (name=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:50.465329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:50.466737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be717f29-a2f1-4ca8-a090-2c5a0168ce12 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:50.476492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 514/1006] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:50 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:50.477465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:50.578474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:50.581957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd06413e-4cb7-46f4-9a76-76479265dabd None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:50.616072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:50.616766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:50.617050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:50.617326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:50.618037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] RBAC: Adding query filter params (type=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:50.618344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:50.619753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5366f60e-fff5-4b55-ac79-f8712d8b7cc9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:50.632676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 493/1007] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:50 2022] GET /identity/v3/services?type=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:51.210533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82d265-f162-483c-8722-a60bb2c93e17 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:51.211045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82d265-f162-483c-8722-a60bb2c93e17 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:51.211123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c82d265-f162-483c-8722-a60bb2c93e17 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:51.220447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 515/1008] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:51.226141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:51.226576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:51.226762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:51.244614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:51.245806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:51.318616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 494/1009] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:51 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:51.343600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82d265-f162-483c-8722-a60bb2c93e17 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:51.389019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:51.450002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c82d265-f162-483c-8722-a60bb2c93e17 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:51.486417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fbda4b-4fc0-447b-b76c-098696147d51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:51.487221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbda4b-4fc0-447b-b76c-098696147d51 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:51.487509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbda4b-4fc0-447b-b76c-098696147d51 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:51.487789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fbda4b-4fc0-447b-b76c-098696147d51 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:51.488990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbda4b-4fc0-447b-b76c-098696147d51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:51.490421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fbda4b-4fc0-447b-b76c-098696147d51 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:51.496933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 516/1010] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:51.498459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:51.596252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:51.603550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67ef5e7-bef1-4ab5-b655-ea30d29b84ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:51.639319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a243fb93-6444-4434-949a-ba58e727a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:51.640100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243fb93-6444-4434-949a-ba58e727a2b0 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:51.640389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243fb93-6444-4434-949a-ba58e727a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:51.640697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a243fb93-6444-4434-949a-ba58e727a2b0 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:51.641412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243fb93-6444-4434-949a-ba58e727a2b0 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:51.642870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a243fb93-6444-4434-949a-ba58e727a2b0 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:51.760760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 495/1011] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:51 2022] POST /identity/v3/projects => generated 311 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:52.083533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2243d-3d79-4980-ab26-dc410984ded1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.083833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2243d-3d79-4980-ab26-dc410984ded1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.084050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c2243d-3d79-4980-ab26-dc410984ded1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.085498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 517/1012] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:52.090871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.091741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.092138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.108858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:52.110056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:52.207149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 496/1013] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:52 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:52.230696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2243d-3d79-4980-ab26-dc410984ded1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:52.338945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c2243d-3d79-4980-ab26-dc410984ded1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:52.348663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:52.374440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:52.375333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.375841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.376129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.376884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:52.378501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:52.391840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:52.393146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 23 13:10:52.394007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:52.396927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 518/1014] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:52 2022] GET /identity/v3/services/compute => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:52.399242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:52.486372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:52.510668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae66abd-e245-4bd5-8f5e-a408c3ff9384 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:52.546976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:52.547696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.548143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.548607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.549790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:52.550102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:52.551483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:52.556842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 497/1015] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:52 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:52.562281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:52.665859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55155d3-673a-4273-a100-3bb25bc8e06c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:52.699770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:52.700490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.701306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.701764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.702478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:52.702785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:52.704144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab43f4a8-cc2a-4970-9239-a8d72ba29d91 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:52.712817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 519/1016] 10.222.0.70 () {56 vars in 1112 bytes} [Mon May 23 13:10:52 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:52.715283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:52.829121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37cafa08-a602-4f1a-9e98-b9f231d7daf1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:52.863388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:52.864058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.864342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.864643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.865385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:52.866764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:52.874937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:52.933275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406f728-537d-4474-805c-cc2f0fd8c5af None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.933496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406f728-537d-4474-805c-cc2f0fd8c5af None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.933761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9406f728-537d-4474-805c-cc2f0fd8c5af None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.935724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 520/1017] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:52.941595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ad176-1b63-45db-a45e-651a0339c644 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:52.941893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ad176-1b63-45db-a45e-651a0339c644 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:52.942194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ad176-1b63-45db-a45e-651a0339c644 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:52.960553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-541ad176-1b63-45db-a45e-651a0339c644 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:52.961779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-541ad176-1b63-45db-a45e-651a0339c644 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:52.978091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 498/1018] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:52 2022] POST /identity/v3/endpoints => generated 334 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:52.998644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:53.054704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 521/1019] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:52 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:53.080260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:53.202411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d4d086-55a3-4acc-b983-99c710a2caff None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:53.237737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048a890c-0aa4-43b9-aef3-e6cee1a6d897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:53.238469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a890c-0aa4-43b9-aef3-e6cee1a6d897 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:53.238756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a890c-0aa4-43b9-aef3-e6cee1a6d897 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:53.239034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048a890c-0aa4-43b9-aef3-e6cee1a6d897 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:53.239776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a890c-0aa4-43b9-aef3-e6cee1a6d897 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:53.241190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048a890c-0aa4-43b9-aef3-e6cee1a6d897 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:53.326675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 499/1020] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:10:53 2022] POST /identity/v3/services => generated 242 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:54.164137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c95c9a-e80c-4af7-9310-29e34b9b0aaa None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:54.164725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c95c9a-e80c-4af7-9310-29e34b9b0aaa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:54.164725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c95c9a-e80c-4af7-9310-29e34b9b0aaa None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:54.166138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 522/1021] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:54.175368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:54.176136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:54.176136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:54.197380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:54.198589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:54.256913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:54.288612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 500/1022] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:54.314147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c95c9a-e80c-4af7-9310-29e34b9b0aaa None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:54.467570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c95c9a-e80c-4af7-9310-29e34b9b0aaa None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:54.541237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5d9ae0-b5b2-4d3b-b735-3141b5106ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:54.542583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9ae0-b5b2-4d3b-b735-3141b5106ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:54.543083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9ae0-b5b2-4d3b-b735-3141b5106ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:54.543635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5d9ae0-b5b2-4d3b-b735-3141b5106ea9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:54.547446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d9ae0-b5b2-4d3b-b735-3141b5106ea9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:54.550099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5d9ae0-b5b2-4d3b-b735-3141b5106ea9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:54.554672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 523/1023] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:54.562235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:54.689917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343bf043-4a1e-4125-9576-4565f96e5042 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:54.726083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30c30c3-a59f-4cad-8a3f-87c0c5c9501f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:54.726707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c30c3-a59f-4cad-8a3f-87c0c5c9501f None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:54.726995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c30c3-a59f-4cad-8a3f-87c0c5c9501f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:54.727274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30c30c3-a59f-4cad-8a3f-87c0c5c9501f None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:54.728017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c30c3-a59f-4cad-8a3f-87c0c5c9501f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:54.729523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30c30c3-a59f-4cad-8a3f-87c0c5c9501f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:54.852597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 501/1024] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:10:54 2022] POST /identity/v3/users => generated 285 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:10:55.532952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25db5a6-26b4-444c-814f-38a9843dca02 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:55.533469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25db5a6-26b4-444c-814f-38a9843dca02 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:55.533469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25db5a6-26b4-444c-814f-38a9843dca02 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:55.534797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 524/1025] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:55.540283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:55.540732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:55.540900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:55.557721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:55.558934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:55.592247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:55.631108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 502/1026] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:55.656172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25db5a6-26b4-444c-814f-38a9843dca02 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:55.689746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:55.763315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25db5a6-26b4-444c-814f-38a9843dca02 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:55.797384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d70acda-dd1a-48cb-8b6a-9de04f77bfa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:55.798132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70acda-dd1a-48cb-8b6a-9de04f77bfa7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:55.798418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70acda-dd1a-48cb-8b6a-9de04f77bfa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:55.798693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d70acda-dd1a-48cb-8b6a-9de04f77bfa7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:55.808879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d70acda-dd1a-48cb-8b6a-9de04f77bfa7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2d70acda-dd1a-48cb-8b6a-9de04f77bfa7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:55.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:55.818882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:55.824721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 525/1027] 10.222.0.70 () {56 vars in 1120 bytes} [Mon May 23 13:10:55 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:55.827156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:55.833057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ff523-4a03-45e3-bec2-14d88718aa97 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:55.833401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ff523-4a03-45e3-bec2-14d88718aa97 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:55.833682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5ff523-4a03-45e3-bec2-14d88718aa97 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:55.835129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 526/1028] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:10:55.840398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:55.840992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:55.841219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:55.859323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:55.860515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:55.937718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87372440-c50b-45b5-81f0-c6b01ecf9e56 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:55.940045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 527/1029] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:55.963422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:55.975378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:55.976370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:55.976950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:55.977237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:55.977986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:55.978491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:55.980591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:55.986527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:55.991746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 503/1030] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:10:55 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:55.999371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.032109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.071954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ebfac2-fccc-4cb7-84d6-bb2571512382 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.107518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5687cd-3e68-4247-a3f8-64354b08b8b4 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.109058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.109866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.110152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.110428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] PATH_INFO: `/v3/services/image` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.111092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.112469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:56.125182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:56.126450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:56.127648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 528/1031] 10.222.0.70 () {56 vars in 1105 bytes} [Mon May 23 13:10:55 2022] GET /identity/v3/services/image => generated 86 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:56.132316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.152646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.153438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.153891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.154010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.165003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.166509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:56.189784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:56.190719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:56.191607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:56.191607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:56.191607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:56.191607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:56.191607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 504/1032] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:56.196881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.243417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.261271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-667449db-ac30-4f89-b2cf-66257e754a12 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.299137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.299819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.300152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.300431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.301317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] RBAC: Adding query filter params (name=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:56.301628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.303113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.309331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 529/1033] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:56.315044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.319489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.338710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c6c994-af79-40be-a583-396d3b297eb9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.381924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.383016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.383016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.383211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.383837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:56.384141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.385728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.398351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 505/1034] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:56.404331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.423325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9141d80a-b5fb-43f1-b90a-0e4179d6c79c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.462172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.462987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.463278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.463563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.464315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] RBAC: Adding query filter params (type=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:56.464660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.466819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.472873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 530/1035] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:56.478378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.494610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.511903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04558934-8684-4941-8517-ca9b009a3e1f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.549364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.550159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.550439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.550715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.558297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.559799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:10:56.574099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:56.575767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:10:56.577222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:56.581023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 506/1036] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/users/nova => generated 82 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:56.581920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.584902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.603497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae173609-a703-460e-b964-cd193ca38343 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.638205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.639035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.639323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.639602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.640603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] RBAC: Adding query filter params (interface=public, service_id=806803d18dae4b80a5d442898f250574, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:56.640928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.642391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.655185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 531/1037] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/endpoints?service_id=806803d18dae4b80a5d442898f250574&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:56.656069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.725516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531472fd-eb85-465c-908a-847a86ccff3b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.741372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.762157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.763352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.763730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.764160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.765216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:56.765535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.766285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461b47af-929e-4bd9-9aaf-fceab5d73d1c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.767047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.789899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 507/1038] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:56.795676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:56.808119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.808823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.809108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.809414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:56.810343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:56.811699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:56.828794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 532/1039] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/services => generated 1957 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:10:56.862164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:56.908765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b1851d9-21ef-4a02-b577-83283327bce1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:56.970163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-336d28f2-aa20-4dc9-b751-f8707950b4b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:56.970841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d28f2-aa20-4dc9-b751-f8707950b4b1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:56.971133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d28f2-aa20-4dc9-b751-f8707950b4b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:56.971412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-336d28f2-aa20-4dc9-b751-f8707950b4b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.051323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d28f2-aa20-4dc9-b751-f8707950b4b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.052939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-336d28f2-aa20-4dc9-b751-f8707950b4b1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.112155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.155420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 508/1040] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 13:10:57.161187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.197374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdb73f-6b42-40a8-b34f-7232f44cae31 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.197702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdb73f-6b42-40a8-b34f-7232f44cae31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.197953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdb73f-6b42-40a8-b34f-7232f44cae31 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.199356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 509/1041] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:57.205629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.205929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.206212 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.225143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:57.226461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:57.276054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79408fd1-b4f7-481a-85df-0e39d6c9e063 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.303335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 510/1042] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:57 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:10:57.311358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.312167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.312455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.312770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.321928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.324082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.326495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.328628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:10:57.350008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:57.351126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:57.352139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:10:57.352139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:10:57.352139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:10:57.352139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:57.352139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 533/1043] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:57.358158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.430808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.442969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d94ed36-deb8-4726-b76a-ab2a20a8ff15 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.468704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a8d270-346f-4ce0-9dbc-e55b27897ccd None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.481539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.482376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.482663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.482941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.495931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:10:57.504964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:57.505945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:10:57.506785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 511/1044] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:57.513034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.520784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.521467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.521749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.522022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.523005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:57.523758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.525538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.539323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 534/1045] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:57.544921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.602365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.623631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d510e6d-d6ae-49a0-8df0-247fb9dcf2c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.656254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc587ee-e4ea-4df4-966c-35d73b8f3b89 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.658786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.659470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.659748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.660023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.660817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:57.661129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.662519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.671524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 512/1046] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:57.677195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.693646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.701104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.703796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.703796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.703796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.709859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.711436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:10:57.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:57.728079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 535/1047] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:10:57.734894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.794937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b1ce0e-8281-4cec-9156-3a0424dcde1e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.830860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.831731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.832013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.832285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] PATH_INFO: `/v3/users/a83bea5c2b21490cbb907fcc8995f166` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.838346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7d4f5e-4604-4f05-867c-c6442a978deb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:57.846302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] RBAC: Authorizing `identity:get_user(user_id=a83bea5c2b21490cbb907fcc8995f166)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.847748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.849638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 513/1048] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/users/a83bea5c2b21490cbb907fcc8995f166 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:57.852818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.854984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.877994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:57.878682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:57.878940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:57.879211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:57.880002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:57.880310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:57.881775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:57.895628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 536/1049] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:57.901477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:57.933323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:57.969691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff1e4-b879-4168-93f9-2b3be5b7d0c7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:58.010440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:58.011103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:58.011386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:58.011665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:58.018340 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4417d7b-6180-4968-9892-8dd811981ffc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:58.056684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:58.057678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:58.058146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:58.058617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:58.060156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] RBAC: Adding query filter params (role.id=71a1477a2ccb4e32a791c47b59a6ea89, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=9cba93afd3344df298ba353d9994ac37) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:58.060670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:58.062265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:58.069005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 537/1050] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/role_assignments?user.id=9cba93afd3344df298ba353d9994ac37&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=71a1477a2ccb4e32a791c47b59a6ea89 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:58.101871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:58.103367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:58.182657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:58.217045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 514/1051] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:10:57 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 13:10:58.223156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:58.267021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:58.336021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a049c62-262b-430f-9f65-025b3165e79f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:58.370901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731b8bb7-9666-4bd1-9aa9-3b2d8783cdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:58.372387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b8bb7-9666-4bd1-9aa9-3b2d8783cdcb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:58.372712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b8bb7-9666-4bd1-9aa9-3b2d8783cdcb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:58.372991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731b8bb7-9666-4bd1-9aa9-3b2d8783cdcb None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:58.374191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b8bb7-9666-4bd1-9aa9-3b2d8783cdcb None admin] RBAC: Authorizing `identity:get_project(project_id=4eeeb260552e4c8ea0b986b603877af9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:58.375623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731b8bb7-9666-4bd1-9aa9-3b2d8783cdcb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:58.377517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 538/1052] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:10:58 2022] GET /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9 => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:58.383511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:58.486031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410ad749-dc25-48a2-ab50-3a1d136bec8c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:58.520704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:58.521363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:58.521645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:58.521915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:58.522998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=4eeeb260552e4c8ea0b986b603877af9, user.id=a83bea5c2b21490cbb907fcc8995f166) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:58.523306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:58.524783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3153e6cb-99b1-4a44-b913-e91ec4c55788 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:58.536616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 515/1053] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:10:58 2022] GET /identity/v3/role_assignments?user.id=a83bea5c2b21490cbb907fcc8995f166&scope.project.id=4eeeb260552e4c8ea0b986b603877af9&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:59.118732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fe9c6-8e46-4d6d-b6e7-84afd06be729 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:59.119250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fe9c6-8e46-4d6d-b6e7-84afd06be729 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:59.119327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492fe9c6-8e46-4d6d-b6e7-84afd06be729 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:59.120862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 539/1054] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:10:59.126669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:59.127066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:59.127266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:59.145031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:10:59.146227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:10:59.175685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:59.225976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 516/1055] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:10:59 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:10:59.249589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fe9c6-8e46-4d6d-b6e7-84afd06be729 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:59.358493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492fe9c6-8e46-4d6d-b6e7-84afd06be729 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:59.360878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:59.395576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:59.396292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:59.396590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:59.396870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] PATH_INFO: `/v3/services/image` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:59.397774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:59.399120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:10:59.412166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:10:59.413214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 23 13:10:59.414078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:10:59.416656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 540/1056] 10.222.0.70 () {56 vars in 1105 bytes} [Mon May 23 13:10:59 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:10:59.420631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:59.519419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9001bff0-bf90-421a-b64f-8e2aa697fc52 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:59.561296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:59.561963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:59.562245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:59.562519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:59.563232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] RBAC: Adding query filter params (name=image) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:59.563537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:59.564948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:59.574857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 517/1057] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:59 2022] GET /identity/v3/services?name=image => generated 122 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:10:59.576883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:59.688878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d83bafd-1f80-48a0-af06-335d85ff1390 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:59.722963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:59.723631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:59.723914 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:59.724188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:59.724940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] RBAC: Adding query filter params (type=image) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:10:59.725249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:59.726637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0a39a4-a0af-47da-a29c-cac8358a3edd None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:59.732364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 541/1058] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:10:59 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:10:59.738007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:10:59.767976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:10:59.841098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd65a42-0e50-4e34-9a22-b59806d94169 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:10:59.876220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7021fa-012b-4c0f-a8dc-4a490dbb0118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:10:59.876904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7021fa-012b-4c0f-a8dc-4a490dbb0118 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:10:59.877185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7021fa-012b-4c0f-a8dc-4a490dbb0118 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:10:59.877455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7021fa-012b-4c0f-a8dc-4a490dbb0118 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:10:59.878204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7021fa-012b-4c0f-a8dc-4a490dbb0118 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:10:59.879669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7021fa-012b-4c0f-a8dc-4a490dbb0118 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:10:59.982893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 518/1059] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:10:59 2022] POST /identity/v3/endpoints => generated 327 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:00.299946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a50e7-6368-4102-9328-c35a9bcc68a9 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.300463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a50e7-6368-4102-9328-c35a9bcc68a9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.300463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268a50e7-6368-4102-9328-c35a9bcc68a9 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.301805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 542/1060] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:00.307358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.307807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.307911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.326909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:00.328125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:00.402820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 519/1061] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:00.425714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a50e7-6368-4102-9328-c35a9bcc68a9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:00.458119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:00.533245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268a50e7-6368-4102-9328-c35a9bcc68a9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:00.567380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:00.568136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.568417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.568728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.582047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:00.592233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:00.593498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:11:00.594352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:00.596962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 543/1062] 10.222.0.70 () {56 vars in 1120 bytes} [Mon May 23 13:11:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:00.600041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:00.618815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:00.711678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae212627-9592-45aa-829b-842dc9dc825e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:00.748630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:00.749310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.749592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.749864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.750608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:00.750913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:00.752305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29a7e49-4f43-498a-a965-9993e395db43 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:00.766047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 520/1063] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:11:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:00.767409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:00.770652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4003846-6a72-4b98-ac4a-65eeafaa8b03 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.770942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4003846-6a72-4b98-ac4a-65eeafaa8b03 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.771220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4003846-6a72-4b98-ac4a-65eeafaa8b03 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.773045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 521/1064] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:11:00.778743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.779041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.779298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.797179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:00.798420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:00.872977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:00.876606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 522/1065] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:11:00.882108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c2e97f-1aa7-482e-9c32-661a6eb21fe4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:00.897712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:00.900016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:00.918196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:00.919286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:00.919566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:00.919836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:00.930607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:00.932101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:11:00.951704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:00.952712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:00.953666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:11:00.953666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:11:00.953666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:00.953666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:00.953666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 544/1066] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:00.958676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.007406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccda333-b490-454e-a875-d601f52c4f78 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.046585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.047393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.047683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.047970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.058575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:01.063105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57c5aad-6130-4b9f-89a9-33592e9fdc9c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:01.067255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:01.068197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:01.073935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 523/1067] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:01.076708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.103283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.103945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.104229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.104515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.105259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:01.105567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.107556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.119483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 545/1068] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:01.125306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.128845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:01.150770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:01.195490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5d30d-a4f2-4849-bc7c-4cdc66770d35 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.234721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.235396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.235896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.236364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.237402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:01.237854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.239392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.239610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d751af-da6f-4fe8-b017-757e6f0332f7 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.250700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 524/1069] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:01.256587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.275774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.276595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.276877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.277146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.285567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.287021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:11:01.302474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:01.303927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:01.304944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 546/1070] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/users/nova => generated 82 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:01.309797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.363981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6010b002-9ef2-40b0-a50a-f0d752438c60 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.374768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:01.390611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:01.401464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.402613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.403805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.404154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] PATH_INFO: `/v3/users/a83bea5c2b21490cbb907fcc8995f166` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.405474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] RBAC: Authorizing `identity:get_user(user_id=a83bea5c2b21490cbb907fcc8995f166)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.407130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.408966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 525/1071] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/users/a83bea5c2b21490cbb907fcc8995f166 => generated 285 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:01.414508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.418764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80371807-e0f0-49e6-8d20-cf4480878d07 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.457730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.458787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.459076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.459358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.460207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:01.460517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.461980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.480792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 547/1072] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:01.486517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.520941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1ffc5e-fa46-4e03-9b64-bff09baa9f97 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.557686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.558549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.559590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.560015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.561428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] RBAC: Authorizing `identity:get_project(project_id=4eeeb260552e4c8ea0b986b603877af9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.563033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.565011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 526/1073] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:01.570699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.595335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6748c9b-2670-4c9b-8582-d01ce1a026d9 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.620547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:01.632924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.633720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.633995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.634267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.640508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:01.652959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.654697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:11:01.674952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:01.676154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:01.677364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:11:01.677364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:11:01.677364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:01.677364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:01.679309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 548/1074] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:01.685406 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.687265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e806ba1-0095-4e84-a656-6fb71e21ceb8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.727503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.728885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.729183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.729464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9/users/a83bea5c2b21490cbb907fcc8995f166/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.732299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4eeeb260552e4c8ea0b986b603877af9, user_id=a83bea5c2b21490cbb907fcc8995f166, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.733868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.787776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3049a29-701a-4fbd-8612-d7ed973b2c6e None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.811820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 527/1075] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:11:01 2022] PUT /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9/users/a83bea5c2b21490cbb907fcc8995f166/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:11:01.823758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:01.824529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:01.825540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:01.825925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:01.826789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:01.827104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:01.828607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:01.843051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 549/1076] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:01.849065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:01.962727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec70c0eb-8fd8-4a22-a956-ba9631a207c5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:01.985775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:02.005423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:02.006231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.006517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.006793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.014977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:02.016683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:11:02.033262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:02.034233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:02.035221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 528/1077] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:11:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:02.040293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:02.081620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:02.157224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067399ab-1f2c-4d26-9802-0316ccff42aa None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:02.198072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:02.198825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.199112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.199439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.200310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:02.200655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:02.202098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d48156-0846-4bc7-9855-095da1490902 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:02.214891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 550/1078] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:11:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:02.221724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:02.261347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3800743-c2e0-491d-895a-27fc52dc6b89 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.262049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3800743-c2e0-491d-895a-27fc52dc6b89 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.263231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3800743-c2e0-491d-895a-27fc52dc6b89 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.265181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 551/1079] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:11:02.270346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.270640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.270923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.288794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:02.290278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:02.335794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c21ed83-dc08-4674-9dd2-6c3d4ffef772 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:02.374589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 552/1080] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:11:02.384010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:02.384997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.385281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.385554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/9cba93afd3344df298ba353d9994ac37/roles/71a1477a2ccb4e32a791c47b59a6ea89` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.388406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=9cba93afd3344df298ba353d9994ac37, role_id=71a1477a2ccb4e32a791c47b59a6ea89)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:02.389813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:02.397712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:02.423200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:02.473371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 529/1081] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:11:02 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/9cba93afd3344df298ba353d9994ac37/roles/71a1477a2ccb4e32a791c47b59a6ea89 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:11:02.502712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ae1b94-962c-4edb-af0b-084cf1b15d65 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:02.542563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:02.543311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.543733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.544014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.620413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:02.622013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:02.650200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:02.734183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 553/1082] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:02 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 13:11:02.739954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:02.787217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:02.848470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa43779-ba21-4a8d-a95a-1e2521570837 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:02.883800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:02.884698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:02.884990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:02.885269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:02.896991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:02.898728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-cde83c6d-3c2f-478d-a192-3dab51223fda None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:11:02.917634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:02.918847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:02.919732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:11:02.919732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:11:02.919732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:02.919732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:02.920696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 530/1083] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:02.925501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:03.037101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b01aa-568f-44dc-97d2-2ec66f5efd0c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:03.061061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:03.071892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:03.074871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:03.074871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:03.074871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:03.074871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:03.074871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:03.075543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f607cc9-8b65-4845-9afa-815b57e02150 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:03.096601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 554/1084] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:11:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:04.069677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96ca1-a7db-4342-a935-15780266db45 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.070207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96ca1-a7db-4342-a935-15780266db45 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.070207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb96ca1-a7db-4342-a935-15780266db45 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.071625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 531/1085] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:04.077756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.078177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.078448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.096008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:04.097236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:04.164817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 555/1086] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:04.189727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96ca1-a7db-4342-a935-15780266db45 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:04.283591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:04.292888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb96ca1-a7db-4342-a935-15780266db45 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:04.327963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:04.328709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.328991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.329265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.339757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:04.347978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:04.349009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:04.349858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 532/1087] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:04.355194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:04.455329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a74ccf-01f5-4754-87b3-51bae0a1b014 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:04.469810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:04.489726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:04.490401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.490684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.490957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.491709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:04.492008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:04.493393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:04.502138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 556/1088] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:04.507891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:04.613060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c2d88-18d8-4b98-9b83-442c6b29518b None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:04.647514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85124c4f-fb83-45c6-8d9a-15ee30462418 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:04.648384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85124c4f-fb83-45c6-8d9a-15ee30462418 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.648692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85124c4f-fb83-45c6-8d9a-15ee30462418 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.648964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85124c4f-fb83-45c6-8d9a-15ee30462418 None admin] PATH_INFO: `/v3/users/a83bea5c2b21490cbb907fcc8995f166` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.650165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85124c4f-fb83-45c6-8d9a-15ee30462418 None admin] RBAC: Authorizing `identity:get_user(user_id=a83bea5c2b21490cbb907fcc8995f166)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:04.651615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85124c4f-fb83-45c6-8d9a-15ee30462418 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:04.653629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 533/1089] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:04 2022] GET /identity/v3/users/a83bea5c2b21490cbb907fcc8995f166 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:04.659304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:04.751739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea163-8b99-47b7-b09f-c8d7384cca49 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.752030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea163-8b99-47b7-b09f-c8d7384cca49 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.752314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548ea163-8b99-47b7-b09f-c8d7384cca49 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.753740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 534/1090] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:11:04.758793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.759106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.759390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.760618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f34a72a-4873-47c3-883b-c18c4d80551d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:04.781125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:04.782346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:04.795813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:04.796471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:04.796780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:04.797144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:04.801248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:04.843457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:04.854538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 535/1091] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:11:04.866402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:04.867772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:04.879393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:04.968817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 557/1092] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:04.974840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:04.985239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f704aa-8b83-484e-a13c-aebc4866d664 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.020954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.021778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.022320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.022602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.033061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:05.041384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:05.042331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:05.044709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 536/1093] 10.222.0.70 () {56 vars in 1120 bytes} [Mon May 23 13:11:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:05.049199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:05.052311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:05.062368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:05.075934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6253e6e-d591-4c71-b838-b06797681bd2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.109879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.110742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.111020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.111305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.112529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] RBAC: Authorizing `identity:get_project(project_id=4eeeb260552e4c8ea0b986b603877af9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.114123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.115814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 558/1094] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:04 2022] GET /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:05.122514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:05.156778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f7dc38-a6f6-4dab-bc75-bd24b6fd1df7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.191019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.191688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.191969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.192241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.193022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:05.193331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.194737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.203138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 537/1095] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:05.208947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:05.228694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b268a8-c2b8-460d-a450-ad65171c7b0d None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.264399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.265092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.265374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.265650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.266499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=4eeeb260552e4c8ea0b986b603877af9, user.id=a83bea5c2b21490cbb907fcc8995f166) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:05.266809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.268266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.274679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 559/1096] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/role_assignments?user.id=a83bea5c2b21490cbb907fcc8995f166&scope.project.id=4eeeb260552e4c8ea0b986b603877af9&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:05.282152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:05.315000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-164a2db5-d2cc-4c64-aea8-ed73cdc86cf5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.352508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.353358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.353657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.353937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.367808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.369350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:11:05.388297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:05.389428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:05.390391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:11:05.390391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:11:05.390391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:05.390391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:05.390391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 538/1097] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:05.395661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:05.408431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:05.503687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3180a3de-abaf-4aab-8a3c-b6ccdb17ee58 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.538436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.539108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.539393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.539671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.540399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:05.540756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.542125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.554114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 560/1098] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:05.560404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:05.663170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00319c22-fdb8-44b8-9286-b0a7c0f4f49e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.687133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:05.697690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.698575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.698861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.699138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.707685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.709175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:11:05.723615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:05.724645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:05.725624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 539/1099] 10.222.0.70 () {56 vars in 1093 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:05.731356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:05.797972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:05.836289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21c40ec-9184-4a9c-b162-92d2d512f29a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:05.870140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:05.870821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:05.871162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:05.871441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:05.872305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:05.872645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:05.874077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:05.892236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 561/1100] 10.222.0.70 () {56 vars in 1133 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:05.898272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:06.008415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571bc1e-a572-4451-98f6-da4a17231083 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:06.042249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:06.042938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:06.043770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:06.044059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:06.095917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:06.111882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:06.113384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:06.213867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 540/1101] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:06.220146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:06.320851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5d2e0f-a0e9-4206-b08d-6ee23c9e09a3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:06.358613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:06.367447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:06.368252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:06.376045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:06.376045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:06.380650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:06.382172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:11:06.401671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:06.403220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:06.408894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:11:06.408894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:11:06.408894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:11:06.408894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:06.408894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 562/1102] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:06.409409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:06.493696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:06.510288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f3250-4042-4035-9982-f9bf670b17d1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:06.547850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:06.547850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:06.548094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:06.548281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:06.549014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:06.549328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:06.550716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:06.563399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 541/1103] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:11:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:06.569003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:06.670134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43dd73fc-0170-49b7-abaf-a08f17bae577 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:06.703841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:06.704671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:06.704959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:06.705237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:06.711911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:06.713383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:11:06.728084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:06.729212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:06.730148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 563/1104] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:11:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:06.735401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:06.838903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e861c8b-4df7-42e5-9f20-e557e269e38c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:06.874195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:06.874860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:06.875150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:06.875431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:06.876223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:06.876535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:06.878321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f66bff-44a4-4179-8bdc-ef495a7c02f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:06.895227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 542/1105] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:11:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:06.901722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:06.947900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:07.006730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d6ab64-46c7-41f2-a1ed-fd208a1f01a4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:07.041628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:07.042295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:07.042578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:07.042924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:07.043765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] RBAC: Adding query filter params (role.id=71a1477a2ccb4e32a791c47b59a6ea89, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=9cba93afd3344df298ba353d9994ac37) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:07.044080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:07.045549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7a263fe-1bc4-48a1-a1ab-7695abd5aa91 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:07.051845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 564/1106] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:06 2022] GET /identity/v3/role_assignments?user.id=9cba93afd3344df298ba353d9994ac37&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=71a1477a2ccb4e32a791c47b59a6ea89 => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:07.546260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833725c-2a7f-4c2a-b325-500824b640b2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:07.546804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833725c-2a7f-4c2a-b325-500824b640b2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:07.546804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2833725c-2a7f-4c2a-b325-500824b640b2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:07.548192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 543/1107] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:07.553944 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:07.554371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:07.554501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:07.571748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:07.572986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:07.641026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 565/1108] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:07.666104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833725c-2a7f-4c2a-b325-500824b640b2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:07.671241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:07.779369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2833725c-2a7f-4c2a-b325-500824b640b2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:07.816127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c543fea0-7b9f-41f8-85e2-07cebde224f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:07.816943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543fea0-7b9f-41f8-85e2-07cebde224f6 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:07.817223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543fea0-7b9f-41f8-85e2-07cebde224f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:07.817502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c543fea0-7b9f-41f8-85e2-07cebde224f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:07.818692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543fea0-7b9f-41f8-85e2-07cebde224f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:07.820143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c543fea0-7b9f-41f8-85e2-07cebde224f6 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:07.821820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 544/1109] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:07.828351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:07.859679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:07.929954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b556e148-da95-44d3-84ff-a70b9d171a25 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:07.966212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59d548e-6885-4bb3-8ffb-93380cd10363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:07.966893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d548e-6885-4bb3-8ffb-93380cd10363 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:07.967184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d548e-6885-4bb3-8ffb-93380cd10363 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:07.967462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59d548e-6885-4bb3-8ffb-93380cd10363 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:07.968210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d548e-6885-4bb3-8ffb-93380cd10363 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:07.969681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59d548e-6885-4bb3-8ffb-93380cd10363 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:08.101603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 566/1110] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:11:07 2022] POST /identity/v3/users => generated 286 bytes in 274 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:10.351779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a262d8-e0bd-43a2-af4a-a3e7098e98d3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:10.352321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a262d8-e0bd-43a2-af4a-a3e7098e98d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:10.352321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a262d8-e0bd-43a2-af4a-a3e7098e98d3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:10.353748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 545/1111] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:10.359563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:10.359992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:10.360128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:10.378145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:10.379335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:10.459382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 567/1112] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:10.485208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a262d8-e0bd-43a2-af4a-a3e7098e98d3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:10.554732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:10.583188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a262d8-e0bd-43a2-af4a-a3e7098e98d3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:10.616633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:10.617489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:10.617775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:10.618054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] PATH_INFO: `/v3/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:10.619542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] RBAC: Authorizing `identity:get_role(role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:10.621090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:10.622402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 546/1113] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:10 2022] GET /identity/v3/roles/a7511c9c66e546dda3a737b069282dae => generated 223 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:10.628199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:10.646974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:10.729229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c733ed5-97c8-46bb-bcd8-07017c24dc9a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:10.763567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:10.764438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:10.764771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:10.765052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] PATH_INFO: `/v3/users/69577301bfcb4d57a2a383e3e3e09828` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:10.779043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] RBAC: Authorizing `identity:get_user(user_id=69577301bfcb4d57a2a383e3e3e09828)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:10.780483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:10.782176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 568/1114] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:10 2022] GET /identity/v3/users/69577301bfcb4d57a2a383e3e3e09828 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:10.787937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:10.883909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cad30f-5c0a-462d-ad25-8d877ce4dd17 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:10.919367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:10.920031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:10.920316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:10.920612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:10.931087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:10.986416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:10.987808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:11.087943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 547/1115] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:10 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:11.094280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:11.255907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:11.278265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f996fd4-6a24-40d2-98f4-6bf5a047d92f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:11.312450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8741f038-27f5-400f-8d06-ff62571f239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:11.313338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741f038-27f5-400f-8d06-ff62571f239e None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:11.313626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741f038-27f5-400f-8d06-ff62571f239e None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:11.313896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8741f038-27f5-400f-8d06-ff62571f239e None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:11.315059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741f038-27f5-400f-8d06-ff62571f239e None admin] RBAC: Authorizing `identity:get_project(project_id=4eeeb260552e4c8ea0b986b603877af9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:11.316478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8741f038-27f5-400f-8d06-ff62571f239e None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:11.318194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 569/1116] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:11 2022] GET /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9 => generated 311 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:11.324395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:11.365156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:11.429236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06ed804-6fc8-4620-9f62-9ffeef9dd92d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:11.464047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:11.464809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:11.465283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:11.465561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:11.466383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] RBAC: Adding query filter params (role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=4eeeb260552e4c8ea0b986b603877af9, user.id=69577301bfcb4d57a2a383e3e3e09828) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:11.466701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:11.468131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a07adc-d828-47fd-9db0-3b09ef4f0570 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:11.473579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 548/1117] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:11 2022] GET /identity/v3/role_assignments?user.id=69577301bfcb4d57a2a383e3e3e09828&scope.project.id=4eeeb260552e4c8ea0b986b603877af9&role.id=a7511c9c66e546dda3a737b069282dae => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:13.675783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986d6d-7d59-46b9-bb5a-cc64800f1978 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:13.676308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986d6d-7d59-46b9-bb5a-cc64800f1978 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:13.676397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83986d6d-7d59-46b9-bb5a-cc64800f1978 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:13.677751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 570/1118] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:13.683340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:13.683737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:13.684017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:13.700932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:13.702110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:13.766828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 549/1119] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:13.789434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986d6d-7d59-46b9-bb5a-cc64800f1978 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:13.891823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83986d6d-7d59-46b9-bb5a-cc64800f1978 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:13.926350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:13.927146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:13.927433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:13.927710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] PATH_INFO: `/v3/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:13.929350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] RBAC: Authorizing `identity:get_role(role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:13.930721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:13.932093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 571/1120] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:13 2022] GET /identity/v3/roles/a7511c9c66e546dda3a737b069282dae => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:13.938384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:13.970424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:14.033864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f17140-9caa-4391-9a1e-69543d21ca31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:14.067286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:14.068146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:14.068429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:14.068732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] PATH_INFO: `/v3/users/69577301bfcb4d57a2a383e3e3e09828` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:14.069908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] RBAC: Authorizing `identity:get_user(user_id=69577301bfcb4d57a2a383e3e3e09828)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:14.071330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:14.072967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 550/1121] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:13 2022] GET /identity/v3/users/69577301bfcb4d57a2a383e3e3e09828 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:14.079321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:14.093990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:14.178605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c06dff-f5d1-462c-8bcc-871b4626f075 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:14.213135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f760fa-173e-4773-8cf0-52d39e9ec293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:14.214526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f760fa-173e-4773-8cf0-52d39e9ec293 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:14.215156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f760fa-173e-4773-8cf0-52d39e9ec293 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:14.215271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f760fa-173e-4773-8cf0-52d39e9ec293 None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:14.216358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f760fa-173e-4773-8cf0-52d39e9ec293 None admin] RBAC: Authorizing `identity:get_project(project_id=4eeeb260552e4c8ea0b986b603877af9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:14.217824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f760fa-173e-4773-8cf0-52d39e9ec293 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:14.219519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 572/1122] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:14 2022] GET /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9 => generated 311 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:14.225587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:14.322950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab7ae8f-f2ae-4f38-9ad5-c9c697b8fe0e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:14.324533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:14.357382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dded601b-65eb-4b29-8cd4-6fc18c621bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:14.358280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded601b-65eb-4b29-8cd4-6fc18c621bb8 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:14.358790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded601b-65eb-4b29-8cd4-6fc18c621bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:14.359096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dded601b-65eb-4b29-8cd4-6fc18c621bb8 None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9/users/69577301bfcb4d57a2a383e3e3e09828/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:14.361705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded601b-65eb-4b29-8cd4-6fc18c621bb8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4eeeb260552e4c8ea0b986b603877af9, user_id=69577301bfcb4d57a2a383e3e3e09828, role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:14.363189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dded601b-65eb-4b29-8cd4-6fc18c621bb8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:14.426072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 551/1123] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:11:14 2022] PUT /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9/users/69577301bfcb4d57a2a383e3e3e09828/roles/a7511c9c66e546dda3a737b069282dae => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:11:16.763123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd96f2-073b-441f-8029-f8427144a5a6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:16.763709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd96f2-073b-441f-8029-f8427144a5a6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:16.763709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bd96f2-073b-441f-8029-f8427144a5a6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:16.765073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 573/1124] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:16.770387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:16.770824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:16.770950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:16.788259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:16.789492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:16.855773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 552/1125] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:16.879026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd96f2-073b-441f-8029-f8427144a5a6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:16.965492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:16.985879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bd96f2-073b-441f-8029-f8427144a5a6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:17.023246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:17.024050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:17.024334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:17.024645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] PATH_INFO: `/v3/roles/a7511c9c66e546dda3a737b069282dae` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:17.026158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] RBAC: Authorizing `identity:get_role(role_id=a7511c9c66e546dda3a737b069282dae)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:17.028125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:17.029598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 574/1126] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:16 2022] GET /identity/v3/roles/a7511c9c66e546dda3a737b069282dae => generated 223 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:17.035589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:17.127028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:17.135319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c894873-63e4-438e-b00b-f0025162f7db None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:17.169912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:17.175628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:17.175628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:17.175628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] PATH_INFO: `/v3/users/69577301bfcb4d57a2a383e3e3e09828` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:17.175628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] RBAC: Authorizing `identity:get_user(user_id=69577301bfcb4d57a2a383e3e3e09828)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:17.175628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:17.175628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 553/1127] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:17 2022] GET /identity/v3/users/69577301bfcb4d57a2a383e3e3e09828 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:17.181636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:17.279711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d993c6f-8604-4096-9389-dabb43b44932 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:17.313124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:17.313782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:17.314062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:17.314334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:17.332414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:17.383135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:17.384669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:17.481859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 575/1128] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 23 13:11:17.487782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:17.591109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38227bb2-d553-4fd4-86c3-9d43a766c3cc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:17.624631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44164266-5657-4d5f-89db-d68f165eca82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:17.625477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44164266-5657-4d5f-89db-d68f165eca82 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:17.625756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44164266-5657-4d5f-89db-d68f165eca82 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:17.626032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44164266-5657-4d5f-89db-d68f165eca82 None admin] PATH_INFO: `/v3/projects/4eeeb260552e4c8ea0b986b603877af9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:17.627208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44164266-5657-4d5f-89db-d68f165eca82 None admin] RBAC: Authorizing `identity:get_project(project_id=4eeeb260552e4c8ea0b986b603877af9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:17.628677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44164266-5657-4d5f-89db-d68f165eca82 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:17.630361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 554/1129] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:17 2022] GET /identity/v3/projects/4eeeb260552e4c8ea0b986b603877af9 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:17.637002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:17.685657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:17.734704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2181-89e8-4d8c-9704-e4004d0decaf None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:17.767183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:17.767817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:17.768093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:17.768365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:17.769234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] RBAC: Adding query filter params (role.id=a7511c9c66e546dda3a737b069282dae, scope.project.id=4eeeb260552e4c8ea0b986b603877af9, user.id=69577301bfcb4d57a2a383e3e3e09828) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:17.769546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:17.771265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64412947-1c92-4232-8801-a7f44a39ef2a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:17.778314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 576/1130] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:17 2022] GET /identity/v3/role_assignments?user.id=69577301bfcb4d57a2a383e3e3e09828&scope.project.id=4eeeb260552e4c8ea0b986b603877af9&role.id=a7511c9c66e546dda3a737b069282dae => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:19.996812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21a4d-6bf7-4b16-9536-8a1c266d611f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:19.997336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21a4d-6bf7-4b16-9536-8a1c266d611f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:19.997336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e21a4d-6bf7-4b16-9536-8a1c266d611f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:19.998718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 555/1131] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:20.003884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:20.004303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:20.004445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:20.021446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:20.022634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:20.087218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 577/1132] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:11:20.111062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21a4d-6bf7-4b16-9536-8a1c266d611f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:20.138939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:20.210945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e21a4d-6bf7-4b16-9536-8a1c266d611f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:20.244806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9fd2b7-9734-4d78-b91a-1a11c711e140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:20.245593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd2b7-9734-4d78-b91a-1a11c711e140 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:20.245878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd2b7-9734-4d78-b91a-1a11c711e140 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:20.246156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9fd2b7-9734-4d78-b91a-1a11c711e140 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:20.247414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd2b7-9734-4d78-b91a-1a11c711e140 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:20.248831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d9fd2b7-9734-4d78-b91a-1a11c711e140 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:20.250455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 556/1133] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:20.256371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:20.299152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:20.354735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf5f06-0260-4d3e-a863-36ad07b11e30 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:20.388698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbaec359-0bf6-4cb7-958e-acbf32216476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:20.389361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaec359-0bf6-4cb7-958e-acbf32216476 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:20.389647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaec359-0bf6-4cb7-958e-acbf32216476 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:20.389925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbaec359-0bf6-4cb7-958e-acbf32216476 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:20.390648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaec359-0bf6-4cb7-958e-acbf32216476 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:20.392072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbaec359-0bf6-4cb7-958e-acbf32216476 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:20.461720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 578/1134] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:11:20 2022] POST /identity/v3/projects => generated 311 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:22.710222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97d5e-8e1a-4c18-860d-740ba6d6e063 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:22.710775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97d5e-8e1a-4c18-860d-740ba6d6e063 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:22.710883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d97d5e-8e1a-4c18-860d-740ba6d6e063 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:22.712176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 557/1135] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:22.717532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:22.717968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:22.718150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:22.736839 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:22.738723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:22.813270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 579/1136] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:22.838911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d97d5e-8e1a-4c18-860d-740ba6d6e063 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:22.941919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:22.948911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d97d5e-8e1a-4c18-860d-740ba6d6e063 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:22.985284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e631c-d49a-4156-8df6-3fa511e481c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:22.986141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e631c-d49a-4156-8df6-3fa511e481c1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:22.986762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e631c-d49a-4156-8df6-3fa511e481c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:22.987042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e631c-d49a-4156-8df6-3fa511e481c1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:22.988238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e631c-d49a-4156-8df6-3fa511e481c1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:22.989726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0e631c-d49a-4156-8df6-3fa511e481c1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:22.991379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 558/1137] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:11:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:22.997768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:23.026033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:23.121721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10015cf3-bb84-41c7-b85d-6e67d397af1f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:23.155187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44dbf77a-85e8-42a1-860e-a30ec7868fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:23.155865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbf77a-85e8-42a1-860e-a30ec7868fb8 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:23.156154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbf77a-85e8-42a1-860e-a30ec7868fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:23.156432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44dbf77a-85e8-42a1-860e-a30ec7868fb8 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:23.157185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbf77a-85e8-42a1-860e-a30ec7868fb8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:23.158615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44dbf77a-85e8-42a1-860e-a30ec7868fb8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:23.247233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 580/1138] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:11:22 2022] POST /identity/v3/users => generated 286 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:25.532741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb07b0-1276-485d-8ac9-654a8c4e6cc2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:25.533271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb07b0-1276-485d-8ac9-654a8c4e6cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:25.533271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb07b0-1276-485d-8ac9-654a8c4e6cc2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:25.534585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 559/1139] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:25.540146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cf855-d788-401f-b173-1b5378e345ca None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:25.540608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cf855-d788-401f-b173-1b5378e345ca None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:25.540763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748cf855-d788-401f-b173-1b5378e345ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:25.559020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-748cf855-d788-401f-b173-1b5378e345ca None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:25.560211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-748cf855-d788-401f-b173-1b5378e345ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:25.632561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 581/1140] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:11:25.656762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb07b0-1276-485d-8ac9-654a8c4e6cc2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:25.761702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb07b0-1276-485d-8ac9-654a8c4e6cc2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:25.794236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:25.794949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:25.795235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:25.795512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:25.805362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:25.813690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:25.814890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:25.816172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 560/1141] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:25.820962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cf855-d788-401f-b173-1b5378e345ca None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:25.832831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:25.930095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748cf855-d788-401f-b173-1b5378e345ca None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:25.964363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:25.965072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:25.965363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:25.965643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:25.966394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:25.966706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:25.968076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:25.976505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 582/1142] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:25.983326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:26.009342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:26.082774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726e7d1a-fe89-45fa-8b27-31f8d5b43c81 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:26.116200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:26.117206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:26.117494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:26.117770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] PATH_INFO: `/v3/users/805b902c2ef84bd1b5e73d51d96b3320` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:26.131556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] RBAC: Authorizing `identity:get_user(user_id=805b902c2ef84bd1b5e73d51d96b3320)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:26.133054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:26.134729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 561/1143] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:25 2022] GET /identity/v3/users/805b902c2ef84bd1b5e73d51d96b3320 => generated 286 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:26.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:26.211694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:26.241956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21356e32-f9e6-4572-b99a-4ceb44b887f1 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:26.275036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:26.276367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:26.276746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:26.277133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:26.347097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:26.348449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:26.414504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:26.443746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 583/1144] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:26.449694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:26.529435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:26.547523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d369822-b406-49eb-bc0e-56d630b0ec89 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:26.580933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ae3301-8770-490a-a129-b2eb01439a7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:26.581790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3301-8770-490a-a129-b2eb01439a7c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:26.582082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3301-8770-490a-a129-b2eb01439a7c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:26.582361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ae3301-8770-490a-a129-b2eb01439a7c None admin] PATH_INFO: `/v3/projects/04e357aec3ad4fb195cfaad82ef63120` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:26.583532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae3301-8770-490a-a129-b2eb01439a7c None admin] RBAC: Authorizing `identity:get_project(project_id=04e357aec3ad4fb195cfaad82ef63120)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:26.585070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ae3301-8770-490a-a129-b2eb01439a7c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:26.587168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 562/1145] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:26 2022] GET /identity/v3/projects/04e357aec3ad4fb195cfaad82ef63120 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:26.600028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:26.700202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac9f645-59b1-49b0-ab40-c434ea16b9d3 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:26.734354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:26.735295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:26.735585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:26.735864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:26.736743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=04e357aec3ad4fb195cfaad82ef63120, user.id=805b902c2ef84bd1b5e73d51d96b3320) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:26.737056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:26.738485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4027fac0-ff9e-4ae0-9738-d259b76c577c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:26.750400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 584/1146] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:26 2022] GET /identity/v3/role_assignments?user.id=805b902c2ef84bd1b5e73d51d96b3320&scope.project.id=04e357aec3ad4fb195cfaad82ef63120&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:28.970007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd173-d4ce-45d7-98c7-7ecfca3a3fe2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:28.970560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd173-d4ce-45d7-98c7-7ecfca3a3fe2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:28.970560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606bd173-d4ce-45d7-98c7-7ecfca3a3fe2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:28.971902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 563/1147] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:28.977357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:28.977860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:28.977955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:28.996003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:28.997249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:29.028274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:29.067008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 585/1148] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:29.090854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bd173-d4ce-45d7-98c7-7ecfca3a3fe2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:29.192406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606bd173-d4ce-45d7-98c7-7ecfca3a3fe2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:29.227878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:29.229413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:29.229696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:29.229969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:29.241167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:29.249064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:29.250173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:29.251056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 564/1149] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:29.256278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:29.351921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf3432-be50-4c21-a1ed-ad67af28c9a9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:29.386222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:29.386883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:29.387162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:29.387433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:29.388173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:29.388476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:29.389856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:29.398056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 586/1150] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:29.404630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:29.418020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:29.503431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-808c87c8-b4ed-40fe-87b9-35dde4a1e5e8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:29.537476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:29.538349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:29.538635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:29.538911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] PATH_INFO: `/v3/users/805b902c2ef84bd1b5e73d51d96b3320` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:29.540101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] RBAC: Authorizing `identity:get_user(user_id=805b902c2ef84bd1b5e73d51d96b3320)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:29.542143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:29.543781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 565/1151] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:29 2022] GET /identity/v3/users/805b902c2ef84bd1b5e73d51d96b3320 => generated 286 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:29.549570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:29.579329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:29.653646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1900ec6a-f3da-4c38-b08d-e979e065ed26 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:29.686710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64313ef5-f5f5-4445-a008-eca9680bd758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:29.687579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64313ef5-f5f5-4445-a008-eca9680bd758 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:29.687863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64313ef5-f5f5-4445-a008-eca9680bd758 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:29.688141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64313ef5-f5f5-4445-a008-eca9680bd758 None admin] PATH_INFO: `/v3/projects/04e357aec3ad4fb195cfaad82ef63120` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:29.689337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64313ef5-f5f5-4445-a008-eca9680bd758 None admin] RBAC: Authorizing `identity:get_project(project_id=04e357aec3ad4fb195cfaad82ef63120)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:29.690790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64313ef5-f5f5-4445-a008-eca9680bd758 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:29.692493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 587/1152] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:29 2022] GET /identity/v3/projects/04e357aec3ad4fb195cfaad82ef63120 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:29.699072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:29.778326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:29.799176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb1481-7f9e-4fbc-aab2-0f4a167eaccc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:29.841461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b061179-ee39-4330-a2ae-864355aeecf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:29.842366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b061179-ee39-4330-a2ae-864355aeecf9 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:29.842655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b061179-ee39-4330-a2ae-864355aeecf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:29.842943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b061179-ee39-4330-a2ae-864355aeecf9 None admin] PATH_INFO: `/v3/projects/04e357aec3ad4fb195cfaad82ef63120/users/805b902c2ef84bd1b5e73d51d96b3320/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:29.844955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b061179-ee39-4330-a2ae-864355aeecf9 None admin] RBAC: Authorizing `identity:create_grant(project_id=04e357aec3ad4fb195cfaad82ef63120, user_id=805b902c2ef84bd1b5e73d51d96b3320, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:29.846406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b061179-ee39-4330-a2ae-864355aeecf9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:29.923360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 566/1153] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:11:29 2022] PUT /identity/v3/projects/04e357aec3ad4fb195cfaad82ef63120/users/805b902c2ef84bd1b5e73d51d96b3320/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:11:32.147771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfade226-49de-444b-b6a8-83c9e6659cc2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:32.148347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfade226-49de-444b-b6a8-83c9e6659cc2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:32.148347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfade226-49de-444b-b6a8-83c9e6659cc2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:32.149713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 588/1154] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:32.155319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:32.155821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:32.155918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:32.173171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:32.174363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:32.242050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 567/1155] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:32.265321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfade226-49de-444b-b6a8-83c9e6659cc2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:32.361210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:32.367727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfade226-49de-444b-b6a8-83c9e6659cc2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:32.402679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:32.403464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:32.403745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:32.404020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:32.414415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:32.423353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:32.424499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:32.425577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 589/1156] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:32.430628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:32.496457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:32.529097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab4886a-1275-410b-84dc-302e0a11bb73 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:32.564764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:32.565435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:32.565720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:32.565999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:32.566747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:32.567055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:32.568431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:32.577119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 568/1157] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:32.582963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:32.684598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3623e9c6-6040-418d-ac77-dc6b6653ffe5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:32.720050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:32.720950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:32.721234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:32.721509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] PATH_INFO: `/v3/users/805b902c2ef84bd1b5e73d51d96b3320` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:32.722683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] RBAC: Authorizing `identity:get_user(user_id=805b902c2ef84bd1b5e73d51d96b3320)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:32.724103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:32.725753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 590/1158] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:32 2022] GET /identity/v3/users/805b902c2ef84bd1b5e73d51d96b3320 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:32.731814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:32.831149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222ea730-f097-47c5-b8f1-dbcc89449470 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:32.859475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:32.865023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:32.865683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:32.865970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:32.866249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:32.933714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:32.935250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:33.034074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 569/1159] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:33.040069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:33.065997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:33.140065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddad7d-30df-44f1-99de-394880c6ab44 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:33.178378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125e8351-99d3-4676-8da8-d119bdc8a4f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:33.179245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e8351-99d3-4676-8da8-d119bdc8a4f3 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:33.179526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e8351-99d3-4676-8da8-d119bdc8a4f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:33.179797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125e8351-99d3-4676-8da8-d119bdc8a4f3 None admin] PATH_INFO: `/v3/projects/04e357aec3ad4fb195cfaad82ef63120` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:33.181729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e8351-99d3-4676-8da8-d119bdc8a4f3 None admin] RBAC: Authorizing `identity:get_project(project_id=04e357aec3ad4fb195cfaad82ef63120)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:33.182945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125e8351-99d3-4676-8da8-d119bdc8a4f3 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:33.184883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 591/1160] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:33 2022] GET /identity/v3/projects/04e357aec3ad4fb195cfaad82ef63120 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:33.190757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:33.220593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:33.292296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f71c1c-e4b8-43de-8071-68440f18051f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:33.332194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:33.332878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:33.333166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:33.333452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:33.334285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=04e357aec3ad4fb195cfaad82ef63120, user.id=805b902c2ef84bd1b5e73d51d96b3320) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:33.334599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:33.336040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf02738-80a0-4252-8fd1-3ddb7ae6f5c5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:33.341901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 570/1161] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:33 2022] GET /identity/v3/role_assignments?user.id=805b902c2ef84bd1b5e73d51d96b3320&scope.project.id=04e357aec3ad4fb195cfaad82ef63120&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:35.624682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93514e-0872-4817-b346-82ffaaa26f1d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:35.625225 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93514e-0872-4817-b346-82ffaaa26f1d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:35.625225 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c93514e-0872-4817-b346-82ffaaa26f1d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:35.626596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 592/1162] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:35.631785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:35.632164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:35.632349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:35.651378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:35.652628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:35.722020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 571/1163] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:35 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:35.745684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93514e-0872-4817-b346-82ffaaa26f1d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:35.843629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:35.856146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c93514e-0872-4817-b346-82ffaaa26f1d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:35.892098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:35.892692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:35.892983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:35.893259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:35.963224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:35.964706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:36.058932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:36.068979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 593/1164] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:36.075806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:36.100863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:36.185430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c13e6f-070c-4562-9824-79a3f07e72ca None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:36.220448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:36.221274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:36.221557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:36.221831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:36.228861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:36.230305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2cda9c0b-a8b5-4319-8e79-c61ca49fbb03 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:36.244713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:36.245918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:36.246950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 572/1165] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:11:36 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:11:36.253656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:36.363037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f66a36-e253-4e90-802d-cab37c3aa050 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:36.398119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:36.398796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:36.399085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:36.399369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:36.400291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:36.400558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:36.401960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2adc04-a447-414c-b3c7-b4f583604a78 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:36.409897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 594/1166] 10.222.0.70 () {56 vars in 1115 bytes} [Mon May 23 13:11:36 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:38.626311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe211985-ed9c-4c36-8f4c-6f29cc0cf064 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:38.627080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe211985-ed9c-4c36-8f4c-6f29cc0cf064 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:38.627080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe211985-ed9c-4c36-8f4c-6f29cc0cf064 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:38.628219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 573/1167] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:11:38.634103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef88c-30eb-4177-9242-11a4b1e32ef8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:38.634806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef88c-30eb-4177-9242-11a4b1e32ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:38.634806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0ef88c-30eb-4177-9242-11a4b1e32ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:38.659192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fa0ef88c-30eb-4177-9242-11a4b1e32ef8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:38.660495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fa0ef88c-30eb-4177-9242-11a4b1e32ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:38.713118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:38.729889 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 595/1168] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:38.753447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe211985-ed9c-4c36-8f4c-6f29cc0cf064 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:38.846693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:38.862878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe211985-ed9c-4c36-8f4c-6f29cc0cf064 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:38.900171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15202dcd-004b-4bfa-add0-4d05fb1b0e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:38.900776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15202dcd-004b-4bfa-add0-4d05fb1b0e58 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:38.901060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15202dcd-004b-4bfa-add0-4d05fb1b0e58 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:38.901332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15202dcd-004b-4bfa-add0-4d05fb1b0e58 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:38.902186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15202dcd-004b-4bfa-add0-4d05fb1b0e58 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:38.903582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15202dcd-004b-4bfa-add0-4d05fb1b0e58 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:39.041992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 574/1169] 10.222.0.70 () {60 vars in 1138 bytes} [Mon May 23 13:11:38 2022] POST /identity/v3/domains => generated 267 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:41.299494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c47b5e-ff27-4d84-9a11-d5502f74031b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:41.300026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c47b5e-ff27-4d84-9a11-d5502f74031b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:41.300026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c47b5e-ff27-4d84-9a11-d5502f74031b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:41.301470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 596/1170] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:41.307533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:41.308006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:41.308091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:41.329197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:41.331175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:41.402307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 575/1171] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:41.426016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c47b5e-ff27-4d84-9a11-d5502f74031b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:41.530202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c47b5e-ff27-4d84-9a11-d5502f74031b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:41.563617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8991c0ee-0855-496f-94a4-2e6919f18706 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:41.564489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991c0ee-0855-496f-94a4-2e6919f18706 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:41.564802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991c0ee-0855-496f-94a4-2e6919f18706 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:41.565080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8991c0ee-0855-496f-94a4-2e6919f18706 None admin] PATH_INFO: `/v3/domains/5b9b351ddeca43fb814b1214e2134015` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:41.575827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991c0ee-0855-496f-94a4-2e6919f18706 None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b9b351ddeca43fb814b1214e2134015)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:41.577297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8991c0ee-0855-496f-94a4-2e6919f18706 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:41.578927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 597/1172] 10.222.0.70 () {56 vars in 1183 bytes} [Mon May 23 13:11:41 2022] GET /identity/v3/domains/5b9b351ddeca43fb814b1214e2134015 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:41.584752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:41.659467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:41.683604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f210239-171b-46f3-8642-c5bfbc78f7dd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:41.718326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c94ff7-5d73-49f0-9d0c-7afc52569620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:41.718981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94ff7-5d73-49f0-9d0c-7afc52569620 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:41.719264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94ff7-5d73-49f0-9d0c-7afc52569620 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:41.719542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c94ff7-5d73-49f0-9d0c-7afc52569620 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:41.720243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c94ff7-5d73-49f0-9d0c-7afc52569620 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:41.721707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76c94ff7-5d73-49f0-9d0c-7afc52569620 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:41.837783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 576/1173] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:11:41 2022] POST /identity/v3/projects => generated 361 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:44.112971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca793a5-083d-4c35-ada4-6e1d27ec5c96 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:44.113532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca793a5-083d-4c35-ada4-6e1d27ec5c96 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:44.113532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca793a5-083d-4c35-ada4-6e1d27ec5c96 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:44.114888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 598/1174] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:44.120859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:44.121303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:44.121424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:44.138162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:44.139389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:44.207344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 577/1175] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:44.234149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca793a5-083d-4c35-ada4-6e1d27ec5c96 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:44.282818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:44.338493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca793a5-083d-4c35-ada4-6e1d27ec5c96 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:44.371872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8218ecc4-cb2e-4ecc-b0c7-7a7e146a0a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:44.372698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218ecc4-cb2e-4ecc-b0c7-7a7e146a0a6c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:44.372987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218ecc4-cb2e-4ecc-b0c7-7a7e146a0a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:44.373268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8218ecc4-cb2e-4ecc-b0c7-7a7e146a0a6c None admin] PATH_INFO: `/v3/domains/5b9b351ddeca43fb814b1214e2134015` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:44.374446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218ecc4-cb2e-4ecc-b0c7-7a7e146a0a6c None admin] RBAC: Authorizing `identity:get_domain(domain_id=5b9b351ddeca43fb814b1214e2134015)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:44.375883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8218ecc4-cb2e-4ecc-b0c7-7a7e146a0a6c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:44.377539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 599/1176] 10.222.0.70 () {56 vars in 1183 bytes} [Mon May 23 13:11:44 2022] GET /identity/v3/domains/5b9b351ddeca43fb814b1214e2134015 => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:44.383447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:44.437518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:44.481379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6108b3e5-484e-439b-8a6e-815743e9ee8d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:44.514394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f299b-bd01-42d8-b5b0-8116cf262901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:44.515059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f299b-bd01-42d8-b5b0-8116cf262901 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:44.515345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f299b-bd01-42d8-b5b0-8116cf262901 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:44.515621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f299b-bd01-42d8-b5b0-8116cf262901 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:44.516320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f299b-bd01-42d8-b5b0-8116cf262901 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:44.517800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f299b-bd01-42d8-b5b0-8116cf262901 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:44.612379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 578/1177] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:11:44 2022] POST /identity/v3/users => generated 311 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:11:46.934594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13711fbe-11ee-4760-8f64-1714d211717b None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:46.935191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13711fbe-11ee-4760-8f64-1714d211717b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:46.935191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13711fbe-11ee-4760-8f64-1714d211717b None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:46.936512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 600/1178] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:46.941877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:46.942348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:46.942454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:46.959789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:46.961030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:47.026794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 579/1179] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:47.050562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13711fbe-11ee-4760-8f64-1714d211717b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:47.151888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13711fbe-11ee-4760-8f64-1714d211717b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:47.166172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:47.186691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:47.187422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:47.187710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:47.187988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:47.198887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:47.206485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:47.207662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:47.208874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 601/1180] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:47.214173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:47.255843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:47.311831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ae877b-da99-4091-b0e2-3cd6fdbee4a0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:47.345585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:47.346251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:47.346533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:47.346803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:47.347546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:47.347852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:47.349254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:47.357538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 580/1181] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:47.363384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:47.460780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2fcdf-6ac1-4ea3-8906-e0861bca0234 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:47.493593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:47.494464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:47.494746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:47.495022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] PATH_INFO: `/v3/users/002317399a8345bb8f4253f7fb3b176f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:47.507915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] RBAC: Authorizing `identity:get_user(user_id=002317399a8345bb8f4253f7fb3b176f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:47.509456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:47.511130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 602/1182] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:47 2022] GET /identity/v3/users/002317399a8345bb8f4253f7fb3b176f => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:47.516916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:47.619253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91f573a-4771-4930-8a3d-c6e9f95a79df None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:47.621575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:47.652822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:47.653471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:47.653758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:47.654036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:47.719968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:47.721381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:47.816962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 581/1183] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:47 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:47.823429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:47.854138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:47.921754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4854d9-8fb6-4f61-84a7-6058b5a8d4d5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:47.955034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ef1c6d-f4aa-4547-9568-2b7d69bba1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:47.955895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef1c6d-f4aa-4547-9568-2b7d69bba1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:47.956180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef1c6d-f4aa-4547-9568-2b7d69bba1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:47.956457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ef1c6d-f4aa-4547-9568-2b7d69bba1b8 None admin] PATH_INFO: `/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:47.957673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef1c6d-f4aa-4547-9568-2b7d69bba1b8 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8ee8df6adf41bda8a0d733bc3cd7ca)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:47.959095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ef1c6d-f4aa-4547-9568-2b7d69bba1b8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:47.960809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 603/1184] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:47 2022] GET /identity/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca => generated 361 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:47.967045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:47.970280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:48.062815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8c3b31-0fe1-40f2-a4be-b51316d4021f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:48.096158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:48.096878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:48.097187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:48.097466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:48.098394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=bc8ee8df6adf41bda8a0d733bc3cd7ca, user.id=002317399a8345bb8f4253f7fb3b176f) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:48.098709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:48.100227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81ec8f-249b-429e-8bc9-486f7c3b3f55 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:48.105736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 582/1185] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:47 2022] GET /identity/v3/role_assignments?user.id=002317399a8345bb8f4253f7fb3b176f&scope.project.id=bc8ee8df6adf41bda8a0d733bc3cd7ca&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:50.294397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbb0ee-5b38-4dea-82c7-61fa5171fe16 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:50.294936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbb0ee-5b38-4dea-82c7-61fa5171fe16 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:50.294936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bbb0ee-5b38-4dea-82c7-61fa5171fe16 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:50.296336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 604/1186] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:50.301679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:50.302044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:50.302249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:50.324081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:50.325311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:50.390980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:50.396658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 583/1187] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:50.420439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbb0ee-5b38-4dea-82c7-61fa5171fe16 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:50.515419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:50.519481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bbb0ee-5b38-4dea-82c7-61fa5171fe16 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:50.553246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:50.554033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:50.554313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:50.554781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:50.564484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:50.572747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:50.573674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:11:50.574544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 605/1188] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:50.579838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:50.674416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fd8196-86e8-41ce-b13c-495999dd41ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:50.708755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:50.709643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:50.709963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:50.710236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:50.712035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:50.712347 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:50.713793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:50.721887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 584/1189] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:50.728202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:50.826291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26e9d293-1adc-41ed-be46-253a0643ce50 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:50.859216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:50.860081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:50.860365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:50.860671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] PATH_INFO: `/v3/users/002317399a8345bb8f4253f7fb3b176f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:50.861847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] RBAC: Authorizing `identity:get_user(user_id=002317399a8345bb8f4253f7fb3b176f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:50.863264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:50.864910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 606/1190] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:50 2022] GET /identity/v3/users/002317399a8345bb8f4253f7fb3b176f => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:50.871322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:50.931730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:50.971985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf8a2ea-334b-4b18-a13c-010a25e2089f None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:51.005109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068961b2-d9fd-4b40-aa50-a0d08a5412e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:51.005971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068961b2-d9fd-4b40-aa50-a0d08a5412e8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:51.006263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068961b2-d9fd-4b40-aa50-a0d08a5412e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:51.006540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068961b2-d9fd-4b40-aa50-a0d08a5412e8 None admin] PATH_INFO: `/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:51.008298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068961b2-d9fd-4b40-aa50-a0d08a5412e8 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8ee8df6adf41bda8a0d733bc3cd7ca)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:51.009777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068961b2-d9fd-4b40-aa50-a0d08a5412e8 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:51.011487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 585/1191] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:50 2022] GET /identity/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca => generated 361 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:51.017495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:51.045518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:51.113986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b4295fa-9d4f-49ed-b6ae-601e1732af21 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:51.147522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e58c41f-dafd-4512-b4e1-cf07e7bc902a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:51.148644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58c41f-dafd-4512-b4e1-cf07e7bc902a None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:51.149394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58c41f-dafd-4512-b4e1-cf07e7bc902a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:51.149677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e58c41f-dafd-4512-b4e1-cf07e7bc902a None admin] PATH_INFO: `/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca/users/002317399a8345bb8f4253f7fb3b176f/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:51.151661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58c41f-dafd-4512-b4e1-cf07e7bc902a None admin] RBAC: Authorizing `identity:create_grant(project_id=bc8ee8df6adf41bda8a0d733bc3cd7ca, user_id=002317399a8345bb8f4253f7fb3b176f, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:51.153116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e58c41f-dafd-4512-b4e1-cf07e7bc902a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:51.275898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 607/1192] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:11:51 2022] PUT /identity/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca/users/002317399a8345bb8f4253f7fb3b176f/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:11:53.471663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77c0f-c464-42eb-986e-ae20306bcae9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:53.472184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77c0f-c464-42eb-986e-ae20306bcae9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:53.472184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad77c0f-c464-42eb-986e-ae20306bcae9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:53.473560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 586/1193] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:11:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:11:53.478983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:53.479399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:53.479538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:53.497177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:11:53.498373 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:11:53.573291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 608/1194] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:11:53 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:11:53.596401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad77c0f-c464-42eb-986e-ae20306bcae9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:53.699752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad77c0f-c464-42eb-986e-ae20306bcae9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:53.728461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:53.736936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:53.743529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:53.743529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:53.743529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:53.750526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:11:53.762755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:11:53.763875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:11:53.765029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 587/1195] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:11:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:11:53.771528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:53.791804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:53.878630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03da52a2-7b81-428b-9d93-3b6ec770f3c0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:53.914684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:53.915348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:53.915631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:53.915905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:53.916691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:53.916998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:53.918354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:53.926732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 609/1196] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:11:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:11:53.935453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:54.055089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef418abf-0600-48df-9900-5b5ee93716ea None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:54.089665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:54.090537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:54.090818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:54.091084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] PATH_INFO: `/v3/users/002317399a8345bb8f4253f7fb3b176f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:54.092275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] RBAC: Authorizing `identity:get_user(user_id=002317399a8345bb8f4253f7fb3b176f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:54.093753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:54.095391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 588/1197] 10.222.0.70 () {56 vars in 1177 bytes} [Mon May 23 13:11:53 2022] GET /identity/v3/users/002317399a8345bb8f4253f7fb3b176f => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:54.101417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:54.212608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:54.233835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaec0d2-117a-4692-80fa-bfb9216e6941 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:54.283930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:54.284611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:54.284899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:54.285180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:54.368313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:54.369810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:54.463591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:54.496625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 610/1198] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:11:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:11:54.502375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:54.558428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:11:54.601781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc30f653-0433-4ea0-a311-50cf8c8858e1 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:54.636476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbcb1270-15d7-4c0a-8a6b-9d48813c5813 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:54.637294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb1270-15d7-4c0a-8a6b-9d48813c5813 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:54.637573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb1270-15d7-4c0a-8a6b-9d48813c5813 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:54.637842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbcb1270-15d7-4c0a-8a6b-9d48813c5813 None admin] PATH_INFO: `/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:54.639025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb1270-15d7-4c0a-8a6b-9d48813c5813 None admin] RBAC: Authorizing `identity:get_project(project_id=bc8ee8df6adf41bda8a0d733bc3cd7ca)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:54.640455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbcb1270-15d7-4c0a-8a6b-9d48813c5813 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:54.642198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 589/1199] 10.222.0.70 () {56 vars in 1186 bytes} [Mon May 23 13:11:54 2022] GET /identity/v3/projects/bc8ee8df6adf41bda8a0d733bc3cd7ca => generated 361 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:11:54.649245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:11:54.746339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189f443e-89cb-44f0-857f-dc6030b31a8c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:11:54.783931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:11:54.784623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:11:54.784912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:11:54.785188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:11:54.786027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=bc8ee8df6adf41bda8a0d733bc3cd7ca, user.id=002317399a8345bb8f4253f7fb3b176f) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:11:54.786339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:11:54.787777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b50e69-23a1-409d-8005-b61611a7a5ca None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:11:54.795205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 611/1200] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:11:54 2022] GET /identity/v3/role_assignments?user.id=002317399a8345bb8f4253f7fb3b176f&scope.project.id=bc8ee8df6adf41bda8a0d733bc3cd7ca&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:12:04.631606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=38 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:12:04.791860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=78 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:17.351339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eba74-5059-46d1-8e58-cfdd69bb5c84 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:17.351913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eba74-5059-46d1-8e58-cfdd69bb5c84 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:17.351913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9eba74-5059-46d1-8e58-cfdd69bb5c84 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:17.353851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 590/1201] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:17.359061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:17.359505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:17.359614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:17.377439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:17.378632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:17.446026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 612/1202] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:13:17.471098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9eba74-5059-46d1-8e58-cfdd69bb5c84 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:17.575781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9eba74-5059-46d1-8e58-cfdd69bb5c84 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:17.610033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:17.610824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:17.611112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:17.611392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:17.621450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:17.622904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:17.640763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:17.641911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:17.643109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:17.643109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:17.643109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:17.643109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:17.643109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 591/1203] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:17.648439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:17.770143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2a5c99-d8f8-4509-889e-7ef1fd2c24f8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:17.780103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:17.785832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:17.786496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:17.786781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:17.787057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:17.787776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:17.788081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:17.789476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6ccdc-2650-4cdd-9b4c-5ee2c30c3316 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:17.800355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 613/1204] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:17.806020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:17.859651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:17.906589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f2601-4a8a-46ad-b673-773f09dd7f96 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:17.945116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475f43e7-7803-426c-b699-7d0fe4369a97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:17.945785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f43e7-7803-426c-b699-7d0fe4369a97 None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:17.946072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f43e7-7803-426c-b699-7d0fe4369a97 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:17.946955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475f43e7-7803-426c-b699-7d0fe4369a97 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:17.947649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f43e7-7803-426c-b699-7d0fe4369a97 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:17.949127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475f43e7-7803-426c-b699-7d0fe4369a97 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:18.043955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 592/1205] 10.222.0.70 () {60 vars in 1132 bytes} [Mon May 23 13:13:17 2022] POST /identity/v3/users => generated 251 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:13:20.296486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede0576-d9e0-45df-911c-48f049d693b1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:20.297052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede0576-d9e0-45df-911c-48f049d693b1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:20.297052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ede0576-d9e0-45df-911c-48f049d693b1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:20.298444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 614/1206] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:20.303797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:20.304232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:20.304594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:20.321680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:20.323000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:20.389432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 593/1207] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:20.413482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede0576-d9e0-45df-911c-48f049d693b1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:20.517262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ede0576-d9e0-45df-911c-48f049d693b1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:20.551316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:20.552124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:20.552411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:20.552719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:20.566128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:13:20.575334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:20.577050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:20.578010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 615/1208] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:13:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:20.582876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:20.622457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:20.678997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-375422ef-8c9e-4edb-a585-69448c4d5960 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:20.711527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:20.712474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:20.712786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:20.713061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:20.713805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:20.714111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:20.715476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:20.723219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 594/1209] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:20.729464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:20.780329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:20.828816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387c5721-915f-4ef7-9294-0175c2f69bfc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:20.871884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:20.872734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:20.873021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:20.873294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:20.885142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:20.886615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:20.904443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:20.905617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:20.906794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:20.906794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:20.906794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:20.906794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:20.906794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 616/1210] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:20.911339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:21.005854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:21.008762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9365005-f15a-46a7-b4fa-74e05e9fdf3a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:21.042568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:21.043179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:21.043706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:21.044170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:21.044965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:21.045559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:21.046929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:21.058746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 595/1211] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:21.064548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:21.165799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b777507d-577e-453d-ba26-afea254c3a01 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:21.167383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:21.201991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:21.202845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:21.203135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:21.203462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:21.211702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:21.218701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:13:21.228833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:21.229958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:21.231262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 617/1212] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:21.236093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:21.339435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed88497e-dfe4-4397-b107-5c23a62e113a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:21.375032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:21.375710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:21.375992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:21.376268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:21.377157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:21.377462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:21.378910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:21.396595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 596/1213] 10.222.0.70 () {56 vars in 1143 bytes} [Mon May 23 13:13:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:21.402475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:21.504910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eedf8c4-c621-45e8-a5a9-78ef1bcbe777 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:21.541738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:21.542444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:21.542740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:21.543022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:21.574635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:21.612317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:21.613863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:21.711584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 618/1214] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:13:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:13:21.718506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:21.737993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:21.816330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a796759-6118-4ae6-97dc-94c5ba55b040 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:21.848822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:21.849608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:21.849891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:21.850163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:21.860502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:21.861987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:21.879466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:21.880586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:21.881709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:21.881709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:21.881709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:21.881709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:21.881709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 597/1215] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:13:21.886735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:21.949917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:21.988456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4867850-e02c-448a-b744-c2e76c8a7b3c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:22.020337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:22.021027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:22.021568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:22.021845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:22.022564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:22.022873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:22.024234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:22.036423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 619/1216] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:22.042830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:22.112128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:22.143071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad4990-63b7-461e-99d3-ae303046b8d9 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:22.176802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:22.177612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:22.177903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:22.178185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:22.185781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:22.187183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:13:22.201873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:22.203681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:22.205383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 598/1217] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:22.208899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:22.310747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053e71dc-854a-4001-b50d-47ad9d3884bc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:22.333792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:22.345707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:22.346487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:22.346774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:22.347053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:22.347840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:22.348148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:22.349594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ae2dda7-e07c-4fc5-862d-a7cd94a63c24 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:22.361713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 620/1218] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:22.367765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:22.467011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a76513-e830-41e8-a4a3-31301bf3ee4e None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:22.494503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:22.500125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:22.500826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:22.501108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:22.501384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:22.502208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=f40f1ccccf2f4b8e9c8373f984be3ddf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:22.502518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:22.503959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4b4aa4-29b5-4cb5-880b-ae7777844c70 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:22.511204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 599/1219] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:13:22 2022] GET /identity/v3/role_assignments?user.id=f40f1ccccf2f4b8e9c8373f984be3ddf&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:24.684787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0b893-b8e7-4a3c-a4a6-ffc0ab2f5991 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:24.685333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0b893-b8e7-4a3c-a4a6-ffc0ab2f5991 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:24.685433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b0b893-b8e7-4a3c-a4a6-ffc0ab2f5991 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:24.686768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 621/1220] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:24.692161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:24.692626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:24.692757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:24.709873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:24.711057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:24.777022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 600/1221] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:24.800142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0b893-b8e7-4a3c-a4a6-ffc0ab2f5991 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:24.904124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b0b893-b8e7-4a3c-a4a6-ffc0ab2f5991 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:24.941255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:24.941980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:24.942262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:24.942536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:24.952818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:13:24.960739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:24.962471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:24.963708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 622/1222] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:13:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:24.968787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:25.067368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d45f3e-da85-4c48-9255-111b92854cdb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:25.091285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:25.102005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:25.102729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:25.103018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:25.103291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:25.104051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] RBAC: Adding query filter params (name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:25.104351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:25.105772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:25.114253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 601/1223] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:13:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:25.120246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:25.164547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:25.221545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d1d667-a3ac-492f-8c59-f2ae60b30d1c None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:25.272211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:25.273036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:25.273322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:25.273597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:25.291558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:25.293068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:25.310307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:25.311446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:25.312371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:25.312371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:25.312371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:25.312371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:25.312371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 623/1224] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:25.317460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:25.416718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02f5b7-1e2c-497c-93eb-6366627bb1bc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:25.451385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:25.452052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:25.452338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:25.452640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:25.453365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:25.453672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:25.455047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:25.467049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 602/1225] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:25.476466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:25.572394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:25.578939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5afd89d-9416-49bf-b790-ac8025fc1b4f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:25.611932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:25.612762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:25.613046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:25.613319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:25.622164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:25.625218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:13:25.640259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:25.641407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:25.642561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 624/1226] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:25.647844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:25.697031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:25.748212 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d6600a-93ef-4602-902a-8ba7a3e8e98c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:25.782026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:25.782698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:25.782982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:25.783257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:25.784085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:25.784394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:25.786068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:25.803211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 603/1227] 10.222.0.70 () {56 vars in 1143 bytes} [Mon May 23 13:13:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:25.810107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:25.910824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a329a15d-4ba4-4b6a-b057-43d418681e7a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:25.953824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:25.954939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:25.955226 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:25.955918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:25.966377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:25.967822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:25.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:25.987093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:25.988284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:25.988284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:25.988284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:25.988284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:25.988284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 625/1228] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:25.993097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:26.105676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81733e7-c2ea-4f77-b9d0-d4feabf15535 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:26.108016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:26.138884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:26.139551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:26.139839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:26.140121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:26.140875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:26.141189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:26.142571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:26.153844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 604/1229] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:26.159522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:26.168882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:26.265201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67c5d4ce-4c1f-4f6e-a316-d060b2729a95 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:26.300277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:26.301096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:26.301383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:26.301659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:26.308615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:26.310060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:13:26.324068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:26.325152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:26.326441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 626/1230] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:13:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:26.331167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:26.432792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e10c8b-6454-4e73-af29-e6d42cac589a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:26.472150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:26.472839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:26.473123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:26.473397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:26.474186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:26.474494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:26.475952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a2ca9-182a-4d2d-83e2-17fb0c5a8643 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:26.487854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 605/1231] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:13:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:26.494437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:26.559224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:26.592843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd776e5a-1d61-486c-9859-01ee875de1a8 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:26.626259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c58273-0d1c-4c48-b513-ec89e486594f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:26.627163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c58273-0d1c-4c48-b513-ec89e486594f None admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:26.627456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c58273-0d1c-4c48-b513-ec89e486594f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:26.627735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c58273-0d1c-4c48-b513-ec89e486594f None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/f40f1ccccf2f4b8e9c8373f984be3ddf/roles/a1ab6b81f077454e952d4133764c7863` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:26.642745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c58273-0d1c-4c48-b513-ec89e486594f None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=f40f1ccccf2f4b8e9c8373f984be3ddf, role_id=a1ab6b81f077454e952d4133764c7863)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:26.644182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44c58273-0d1c-4c48-b513-ec89e486594f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:26.782424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 627/1232] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:13:26 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/f40f1ccccf2f4b8e9c8373f984be3ddf/roles/a1ab6b81f077454e952d4133764c7863 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:13:29.013034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095a5a-e0b0-47bb-be62-35ed4f7b27d9 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.013595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095a5a-e0b0-47bb-be62-35ed4f7b27d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.013595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57095a5a-e0b0-47bb-be62-35ed4f7b27d9 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.015111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 606/1233] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:29.020690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.021074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.021496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.040928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:29.042119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:29.109057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 628/1234] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:29.134066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095a5a-e0b0-47bb-be62-35ed4f7b27d9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:29.161795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:29.253692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57095a5a-e0b0-47bb-be62-35ed4f7b27d9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:29.302210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:29.302962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.303304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.303684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.321837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:13:29.335493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:29.338162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:29.339789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 607/1235] 10.222.0.70 () {56 vars in 1102 bytes} [Mon May 23 13:13:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:29.344706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:29.396052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:29.449638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf99ec4-458a-4c25-ba56-1b33f6db4e85 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:29.486247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:29.486918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.487199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.487473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.488221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] RBAC: Adding query filter params (name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:29.488526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:29.489909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:29.498534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 629/1236] 10.222.0.70 () {56 vars in 1103 bytes} [Mon May 23 13:13:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:29.504354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:29.588952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:29.604908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c9a992-749d-4322-821e-bf573b44d7b8 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:29.640381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:29.641224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.641505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.641777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.653366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:29.655412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:29.674071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:29.675213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:29.676323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:29.676323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:29.676323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:29.676323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:29.680123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 608/1237] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:29.682380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:29.783412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94b171-fc2e-4805-b358-93195156a59f None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:29.793241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:29.819580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:29.820266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.820555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.820865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.822560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:29.822560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:29.823823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:29.836601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 630/1238] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:29.843269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:29.951969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9d9e72-858a-43bf-990b-55d38c3cbee7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:29.989733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:29.990529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:29.990807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:29.991083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:29.997185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:30.002101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:30.003619 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:13:30.019729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:30.021881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:30.023373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 609/1239] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:30.028832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:30.146173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afc8190-a4de-483d-b3bc-95b192816bfc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:30.182412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:30.183099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:30.183393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:30.183672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:30.184870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:30.184870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:30.186372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:30.205852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 631/1240] 10.222.0.70 () {56 vars in 1143 bytes} [Mon May 23 13:13:30 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:30.211736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:30.319375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170b50f1-b164-4fbf-8b64-789440d6a230 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:30.358801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:30.359462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:30.359752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:30.360031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:30.425565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:30.433647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:30.435121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:30.538294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 610/1241] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:13:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:13:30.544990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:30.577021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:30.650411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb1f3ab-6720-4f15-a9a2-16fc4e0a2560 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:30.687260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:30.687981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:30.688257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:30.688527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:30.699106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:30.700555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:30.718980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:30.720181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:30.721160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:30.721160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:30.721160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:30.721160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:30.721160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 632/1242] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:30.726284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:30.827886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:30.833689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50ac10d-fad5-470a-a25d-ae5a6033ac72 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:30.869716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:30.872687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:30.872687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:30.872687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:30.872687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:30.872687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:30.873311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:30.885007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 611/1243] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:30.890775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:30.975877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:31.000637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9bc8ea-d12a-4425-a57e-9d0d98a772ee None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:31.037002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:31.037810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:31.038098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:31.038376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:31.045844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:31.047296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:13:31.061756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:31.062861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:31.063878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 633/1244] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:13:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:31.069137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:31.179342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8fa6c-aecf-4313-9ff6-c7b05fa37132 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:31.218450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:31.219123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:31.219415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:31.219695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:31.220493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:31.220839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:31.222357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-688b7bda-fe40-4147-998d-248da8480372 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:31.234995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 612/1245] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:13:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:31.241311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:31.346700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f2ff3c-4819-48c9-a178-d504ed70d433 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:31.385925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:31.386573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:31.386853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:31.387126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:31.387952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] RBAC: Adding query filter params (role.id=a1ab6b81f077454e952d4133764c7863, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=f40f1ccccf2f4b8e9c8373f984be3ddf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:31.388261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:31.389709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832e6b78-72fa-4526-a07b-85789f691d13 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:31.395527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 634/1246] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:13:31 2022] GET /identity/v3/role_assignments?user.id=f40f1ccccf2f4b8e9c8373f984be3ddf&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=a1ab6b81f077454e952d4133764c7863 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:33.779054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9dcbc-4f75-4660-9803-b69cdc331dbb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:33.779655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9dcbc-4f75-4660-9803-b69cdc331dbb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:33.779655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9dcbc-4f75-4660-9803-b69cdc331dbb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:33.781088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 613/1247] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:33.786271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:33.786650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:33.786836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:33.792199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:33.804154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:33.805386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:33.874077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 635/1248] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:33.898081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9dcbc-4f75-4660-9803-b69cdc331dbb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:33.917348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:33.998749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9dcbc-4f75-4660-9803-b69cdc331dbb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:34.042014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:34.043315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:34.043611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:34.043896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:34.060974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:13:34.069071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:34.070112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:34.070971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 614/1249] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:13:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:13:34.078589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:34.182789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71b6152-7d39-4dc0-bc95-d2083b8dbc63 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:34.216743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:34.221744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:34.222430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:34.222717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:34.222997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:34.223752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:34.224064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:34.225467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:34.234345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 636/1250] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:13:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:34.240483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:34.325423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:34.348192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149c76d-4abb-4976-94b4-9a7346affa2a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:34.384817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:34.385619 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:34.385907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:34.386189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:34.397307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:34.398810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:34.416317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:34.417443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:34.418449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:34.418449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:34.418449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:34.418449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:34.418449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 615/1251] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:34.423476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:34.522847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2e195e-fa64-49d1-8253-25f6010c918b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:34.558665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:34.559334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:34.559617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:34.559886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:34.560636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:34.561057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:34.562429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:34.574716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 637/1252] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:34.583538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:34.682282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d89ac5a-7651-4c6b-bc22-468e052c2153 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:34.708224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:34.716613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:34.717419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:34.717725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:34.717994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:34.725832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:34.727326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:13:34.742073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:34.743384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:34.744655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 616/1253] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:34.749108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:34.809052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:34.853873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea52763-7a6f-49b4-9e31-ef0ac890f9c5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:34.887846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:34.888549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:34.889034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:34.889229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:34.890161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:34.890659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:34.892411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:34.911151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 638/1254] 10.222.0.70 () {56 vars in 1143 bytes} [Mon May 23 13:13:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:34.917512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:35.018110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6dc98f3-ad94-4717-8fe9-fac1458dd9ae None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:35.052077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:35.052749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:35.053037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:35.053311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:35.112164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:35.124494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:35.125984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:35.227170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 617/1255] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:13:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:13:35.234295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:35.341437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-279587e0-2046-4318-aff4-00d00d895351 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:35.356090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:35.378117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:35.378749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:35.379036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:35.379313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:35.389438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:35.390896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:35.408935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:35.410000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:35.411011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:35.411011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:35.411011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:35.411011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:35.411011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 639/1256] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:35.416206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:35.505477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:35.517320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcba7c55-9439-441a-9a0c-1b7411f338f5 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:35.551529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:35.552197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:35.552485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:35.552785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:35.553515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:35.553827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:35.555215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:35.566510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 618/1257] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:35.572856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:35.675677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9646747-bd82-4992-af44-5b2063bacd1a None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:35.710240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:35.711042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:35.711328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:35.711649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:35.719304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:35.720931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:13:35.735025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:35.736134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:35.737431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 640/1258] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:13:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:13:35.742306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:35.846757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908377d9-726c-4922-8bdc-39d82173d1ab None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:35.879925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:35.880590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:35.880894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:35.881169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:35.881979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:35.882348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:35.884355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f4d738-37e6-408d-b718-99842723b9fd None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:35.897450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 619/1259] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:13:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:35.904791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:35.932734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:36.006813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d315bef-453c-459a-aa50-4a49b4439fa4 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:36.042421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:36.043088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:36.043378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:36.043661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:36.044496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=f40f1ccccf2f4b8e9c8373f984be3ddf) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:36.044842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:36.046304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed525056-8b27-4a96-9e1d-ce5d342d0d95 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:36.051812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 641/1260] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:13:35 2022] GET /identity/v3/role_assignments?user.id=f40f1ccccf2f4b8e9c8373f984be3ddf&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=33de09388f974cd391e13bb4f2502a52 => generated 259 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:38.295210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d200a5-7ce2-44c5-a3b3-7de277fa515d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:38.295735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d200a5-7ce2-44c5-a3b3-7de277fa515d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:38.295897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d200a5-7ce2-44c5-a3b3-7de277fa515d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:38.297769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 620/1261] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:38.303447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:38.303880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:38.304012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:38.323009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:38.324228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:38.395835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 642/1262] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:38.418953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d200a5-7ce2-44c5-a3b3-7de277fa515d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:38.430174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:38.523084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d200a5-7ce2-44c5-a3b3-7de277fa515d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:38.559012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:38.559816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:38.560102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:38.560379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:38.571311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:13:38.579675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:38.580772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:38.581639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 621/1263] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:38.587512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:38.607082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:38.690224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2cbc5f9-7c1f-4f27-9b0e-574ca534ce1e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:38.729221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:38.729899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:38.730184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:38.730461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:38.731220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:38.731533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:38.732941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:38.741778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 643/1264] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:13:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:38.748872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:38.822105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:38.856595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7183f61-fd07-49b7-b712-da9a2fa8b11c None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:38.897069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:38.897887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:38.898167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:38.898441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:38.909771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:38.911286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:38.928888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:38.934520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:38.935625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:38.935625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:38.935625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:38.935625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:38.935625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 622/1265] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:38.936841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:39.021801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:39.040437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ba0f0e-d7f4-42df-87b5-6ffb38a53abc None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:39.075009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:39.075604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:39.075891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:39.076168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:39.076934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:39.077244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:39.078671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:39.090967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 644/1266] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:39.096712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:39.203872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1dba0f-04cd-42af-ad02-83e2ca6f2e0d None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:39.225142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:39.239314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:39.240134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:39.240414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:39.240731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:39.250070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:39.251552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:13:39.267184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:39.268456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:39.269674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 623/1267] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:39.275358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:39.390651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2258072-e3bd-4f38-b588-df2dc505f409 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:39.427434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:39.428120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:39.428396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:39.428703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:39.429549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:39.429859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:39.431989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:39.441901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:39.451909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 645/1268] 10.222.0.70 () {56 vars in 1143 bytes} [Mon May 23 13:13:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:39.457816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:39.567257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efeb1a85-1551-4e50-a15d-d39941c0b643 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:39.602331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:39.604141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:39.604428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:39.604737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:39.615667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:39.617158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:39.635457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:39.636622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:39.637892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:39.637892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:39.637892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:39.637892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:39.637892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 624/1269] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:39.642523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:39.743937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817e44f-6fc0-419d-aafe-0c123a79931b None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:39.778095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:39.778760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:39.779045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:39.779320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:39.780032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:39.780338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:39.781726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:39.792822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 646/1270] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:39.798668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:39.832815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:39.901924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d4d508-0720-49db-90e8-45f122ba47f7 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:39.936631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:39.937426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:39.937714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:39.937995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:39.944490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:39.946017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:13:39.960669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:39.961763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:39.963400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 625/1271] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:13:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:39.968675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:40.025426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:40.065393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d876ae3d-baf9-4529-b578-653d4d1cb8a5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:40.097546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:40.098201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:40.098483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:40.098753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:40.099532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:40.099837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:40.101318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92433f8-232b-4e2e-b198-67cde6449917 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:40.114241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 647/1272] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:13:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:40.124544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:40.226868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:40.236475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bf03d0-4761-404f-856b-a92cf3b40dbc None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:40.269922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403ceac5-a75e-486b-ac1b-b2918ba5e803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:40.270814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceac5-a75e-486b-ac1b-b2918ba5e803 None admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:40.271100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceac5-a75e-486b-ac1b-b2918ba5e803 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:40.271375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403ceac5-a75e-486b-ac1b-b2918ba5e803 None admin] PATH_INFO: `/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/f40f1ccccf2f4b8e9c8373f984be3ddf/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:40.273676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ceac5-a75e-486b-ac1b-b2918ba5e803 None admin] RBAC: Authorizing `identity:create_grant(project_id=7e4bc8dfc8a242e2b66f47649e4dd59a, user_id=f40f1ccccf2f4b8e9c8373f984be3ddf, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:40.279298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403ceac5-a75e-486b-ac1b-b2918ba5e803 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:40.451200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 626/1273] 10.222.0.70 () {58 vars in 1439 bytes} [Mon May 23 13:13:40 2022] PUT /identity/v3/projects/7e4bc8dfc8a242e2b66f47649e4dd59a/users/f40f1ccccf2f4b8e9c8373f984be3ddf/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:13:42.827205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332d91-d158-4b47-bb04-b3e92e5384ad None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:42.827835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332d91-d158-4b47-bb04-b3e92e5384ad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:42.827835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98332d91-d158-4b47-bb04-b3e92e5384ad None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:42.829234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 648/1274] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:42.835442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:42.836021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:42.836021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:42.855497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:42.856819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:42.947811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 627/1275] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:13:42.972387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98332d91-d158-4b47-bb04-b3e92e5384ad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:43.097052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98332d91-d158-4b47-bb04-b3e92e5384ad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:43.119060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:43.139166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:43.140004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:43.140292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:43.140599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:43.154114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(role_id) May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 23 13:13:43.162387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:43.164273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:43.166629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 649/1276] 10.222.0.70 () {56 vars in 1096 bytes} [Mon May 23 13:13:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:43.172328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:43.269568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:43.291395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000537cf-a001-498b-bfa4-032fbc405fe7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:43.329868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:43.330630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:43.330954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:43.331263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:43.332086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:43.332461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:43.333963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:43.343882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 628/1277] 10.222.0.70 () {56 vars in 1099 bytes} [Mon May 23 13:13:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:43.349880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:43.468943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b450341-8fa8-45fc-88ef-ace61843bc57 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:43.508290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:43.509331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:43.509843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:43.510137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:43.523350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:43.525738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:43.554871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:43.557227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:43.559735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:43.559735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:43.559735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:43.559735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:43.559735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 650/1278] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:43.563462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:43.677099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34b9524-a6a1-4124-b91e-d7cd6016b558 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:43.720808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:43.721527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:43.721811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:43.722086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:43.722921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:43.723230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:43.724670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:43.738277 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 629/1279] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:43.745282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:43.780292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:43.874849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6ae2da-95ac-400c-858f-135a483e1ccb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:43.917227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:43.918056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:43.918344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:43.918624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:43.926479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:43.928005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_user(user_id) May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 23 13:13:43.944007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:43.945219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:43.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 651/1280] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:43 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:43.952375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:43.956484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:44.062370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2ccba3-c552-40a5-a412-6cfdffa1bf69 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:44.101672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:44.102362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:44.103181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:44.103907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:44.104300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:44.104636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:44.106098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:44.126837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 630/1281] 10.222.0.70 () {56 vars in 1143 bytes} [Mon May 23 13:13:43 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:44.132890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:44.268024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:44.289184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7aab0d4-757c-4f15-830a-59aa8ce3eac5 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:44.327709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:44.328368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:44.328690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:44.329020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:44.403495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:44.405020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:44.509360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:44.521388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 652/1282] 10.222.0.70 () {58 vars in 1282 bytes} [Mon May 23 13:13:44 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 23 13:13:44.529215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:44.621251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:44.664449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78ae338-e87a-4cab-9cbb-45ecd89e218a None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:44.701839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:44.702646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:44.702923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:44.703198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:44.716819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:44.718596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 23 13:13:44.739469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:44.740812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:44.742687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 23 13:13:44.742687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 23 13:13:44.742687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 23 13:13:44.742687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:44.742687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 631/1283] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:13:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:13:44.746963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:44.858384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec127ff-12c3-4e5b-b8a8-31402fbf62eb None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:44.917507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:44.918549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:44.918965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:44.919422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:44.920487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:44.920959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:44.922828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:44.937699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 653/1284] 10.222.0.70 () {56 vars in 1109 bytes} [Mon May 23 13:13:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:44.944560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:45.066314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b38fbf1-5b73-49c3-9fc4-18e8c0a95096 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:45.089055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:45.112140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:45.112916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:45.113201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:45.113480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:45.121779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:45.123506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(project_id) May 23 13:13:45.138836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application with Lock( May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:45.140782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: ERROR keystone.server.flask.application May 23 13:13:45.142557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 632/1285] 10.222.0.70 () {56 vars in 1111 bytes} [Mon May 23 13:13:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:45.146801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:45.204328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:45.263520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f76cb2-5830-47d8-878b-101b12587297 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:45.307133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:45.307809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:45.308097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:45.308376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:45.309215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:45.309530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:45.310972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d218d64-b887-494b-bf3a-3686f4cfc161 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:45.324973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 654/1286] 10.222.0.70 () {56 vars in 1148 bytes} [Mon May 23 13:13:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:45.331768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:45.442443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039f8881-64ce-422c-85a8-421bea60e729 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:45.480295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:45.480999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:45.481288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:45.481571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:45.482418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] RBAC: Adding query filter params (role.id=33de09388f974cd391e13bb4f2502a52, scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a, user.id=f40f1ccccf2f4b8e9c8373f984be3ddf) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:45.482737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:45.484294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc23095-5db1-45ef-8689-5d3d408410d7 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:45.491087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 633/1287] 10.222.0.70 () {56 vars in 1374 bytes} [Mon May 23 13:13:45 2022] GET /identity/v3/role_assignments?user.id=f40f1ccccf2f4b8e9c8373f984be3ddf&scope.project.id=7e4bc8dfc8a242e2b66f47649e4dd59a&role.id=33de09388f974cd391e13bb4f2502a52 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:47.839079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55675f5-0157-42fb-a0cc-e1a3fac91639 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:47.839701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55675f5-0157-42fb-a0cc-e1a3fac91639 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:47.839701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55675f5-0157-42fb-a0cc-e1a3fac91639 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:47.841235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 655/1288] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:47.847542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:47.848175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:47.848175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:47.866615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:47.867873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:47.903228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:47.945266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 634/1289] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:47 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:13:47.985592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55675f5-0157-42fb-a0cc-e1a3fac91639 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:48.099480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55675f5-0157-42fb-a0cc-e1a3fac91639 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:48.112513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:48.137678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:48.138501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:48.138788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:48.139060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:48.139721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:48.141148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:48.154836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:48.156814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:48.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 656/1290] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:13:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:48.165639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:48.279671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819ebaa-d989-4bf5-905d-b242965f84ff None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:48.327037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:48.328084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:48.328428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:48.328749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:48.329476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] RBAC: Adding query filter params (name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:48.329787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:48.331224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4beb4c59-2f05-48ed-b5a6-2495a68e5ebf None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:48.337932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 635/1291] 10.222.0.70 () {56 vars in 1116 bytes} [Mon May 23 13:13:48 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:48.346653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:48.478701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3f2d6a-d8e9-4c76-bafe-0ba81dfd2e40 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:48.518418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:48.519249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:48.519610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:48.519985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] PATH_INFO: `/v3/services` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:48.520860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:48.521264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:48.523129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fa04b6-099e-4170-90b1-7d8583472aa5 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:48.529325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 657/1292] 10.222.0.70 () {56 vars in 1116 bytes} [Mon May 23 13:13:48 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:13:50.807617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253614-5dbd-4527-a5fe-5a71eee9ac68 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:50.808258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253614-5dbd-4527-a5fe-5a71eee9ac68 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:50.808258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f253614-5dbd-4527-a5fe-5a71eee9ac68 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:50.810032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 636/1293] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:50.816038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30588ca0-b5c7-4333-a7be-d9c7c20146a0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:50.816652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30588ca0-b5c7-4333-a7be-d9c7c20146a0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:50.816652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30588ca0-b5c7-4333-a7be-d9c7c20146a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:50.835397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-30588ca0-b5c7-4333-a7be-d9c7c20146a0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:50.837580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30588ca0-b5c7-4333-a7be-d9c7c20146a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:50.874609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:50.943197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 658/1294] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 23 13:13:50.967223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253614-5dbd-4527-a5fe-5a71eee9ac68 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:51.011672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:51.088403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f253614-5dbd-4527-a5fe-5a71eee9ac68 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:51.125667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9075f28e-1806-4ba9-abaa-a49aeebb4fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:51.126333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075f28e-1806-4ba9-abaa-a49aeebb4fad None admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:51.126615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075f28e-1806-4ba9-abaa-a49aeebb4fad None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:51.126887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9075f28e-1806-4ba9-abaa-a49aeebb4fad None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:51.127607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075f28e-1806-4ba9-abaa-a49aeebb4fad None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:51.129026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9075f28e-1806-4ba9-abaa-a49aeebb4fad None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:51.223088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 637/1295] 10.222.0.70 () {60 vars in 1141 bytes} [Mon May 23 13:13:50 2022] POST /identity/v3/services => generated 246 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:13:53.513426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417c847-46b0-4c09-9e13-f809eb17ca0a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:53.513969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417c847-46b0-4c09-9e13-f809eb17ca0a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:53.513969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f417c847-46b0-4c09-9e13-f809eb17ca0a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:53.515519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 659/1296] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:53.522240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:53.522850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:53.522850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:53.543380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:53.544639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:53.627481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 638/1297] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:53 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:53.652190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417c847-46b0-4c09-9e13-f809eb17ca0a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:53.794296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f417c847-46b0-4c09-9e13-f809eb17ca0a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:53.840988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:53.841801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:53.842081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:53.842351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:53.843004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:53.844456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:53.859079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:53.861289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:53.863435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 660/1298] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:13:53 2022] GET /identity/v3/services/placement => generated 90 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 23 13:13:53.869592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:53.887722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:53.970512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a389a87f-1be0-45c9-bd86-4e8f846f1910 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:54.007014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:54.007665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:54.007947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:54.008222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:54.009763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:54.010172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:54.011499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:54.016734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 639/1299] 10.222.0.70 () {56 vars in 1116 bytes} [Mon May 23 13:13:53 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:54.023191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:54.054408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:54.122493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fa7e49-3526-4e6e-b3d3-f97dba207af2 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:54.159168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:54.159846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:54.160135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:54.160413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:54.161261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] RBAC: Adding query filter params (interface=public, service_id=171a97103347485cbcfb3ec108cdf27e, region_id=RegionOne) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:54.161580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:54.162987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2dc4fb-a9f5-4aa6-91eb-2dcc6651ccfa None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:54.169435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 661/1300] 10.222.0.70 () {56 vars in 1251 bytes} [Mon May 23 13:13:54 2022] GET /identity/v3/endpoints?service_id=171a97103347485cbcfb3ec108cdf27e&interface=public®ion_id=RegionOne => generated 194 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:54.175322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:54.257173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:54.280017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9140101e-e493-4dfa-8603-28dcab636988 None admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:54.315672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c61e4d-b0c3-4c60-9cdf-7970e9229468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:54.316291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c61e4d-b0c3-4c60-9cdf-7970e9229468 None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:54.316600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c61e4d-b0c3-4c60-9cdf-7970e9229468 None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:54.316889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c61e4d-b0c3-4c60-9cdf-7970e9229468 None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:54.317611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c61e4d-b0c3-4c60-9cdf-7970e9229468 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:54.319100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c61e4d-b0c3-4c60-9cdf-7970e9229468 None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:54.327128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 640/1301] 10.222.0.70 () {56 vars in 1087 bytes} [Mon May 23 13:13:54 2022] GET /identity/v3/services => generated 2191 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:13:56.610101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00fcc8-006f-4276-b8c9-7ad99def2dc2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:56.610664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00fcc8-006f-4276-b8c9-7ad99def2dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:56.610664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc00fcc8-006f-4276-b8c9-7ad99def2dc2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:56.612094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 662/1302] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:13:56.618131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:56.619399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:56.619399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:56.636781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:13:56.638009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:13:56.713620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 641/1303] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:13:56 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 23 13:13:56.737513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00fcc8-006f-4276-b8c9-7ad99def2dc2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:56.840455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:56.847593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc00fcc8-006f-4276-b8c9-7ad99def2dc2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:56.882545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:56.883284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:56.883572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:56.884358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:56.885055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:56.886433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 13:13:56.898828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_service(service_id) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 13:13:56.901188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 13:13:56.907042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 663/1304] 10.222.0.70 () {56 vars in 1117 bytes} [Mon May 23 13:13:56 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 23 13:13:56.908612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:56.986449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:13:57.010120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78c250e5-b73c-414c-870a-dd13adb1856d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:57.045729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:57.046411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:57.046709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:57.046976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] PATH_INFO: `/v3/services` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:57.047699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] RBAC: Adding query filter params (name=placement) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:13:57.048011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:57.049439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb30a5-db9b-4338-ab52-0c224c8a23cb None admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:57.054901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 642/1305] 10.222.0.70 () {56 vars in 1116 bytes} [Mon May 23 13:13:56 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:13:57.061577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:13:57.170409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4407bb-7713-4907-88ae-d0a40b6a9454 None admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:13:57.206028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f006a802-c369-42b6-90b4-cc8d302f11e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:13:57.206700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a802-c369-42b6-90b4-cc8d302f11e6 None admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:13:57.206987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a802-c369-42b6-90b4-cc8d302f11e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:13:57.207266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f006a802-c369-42b6-90b4-cc8d302f11e6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:13:57.208028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006a802-c369-42b6-90b4-cc8d302f11e6 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:13:57.209436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f006a802-c369-42b6-90b4-cc8d302f11e6 None admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:13:57.348716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 664/1306] 10.222.0.70 () {60 vars in 1144 bytes} [Mon May 23 13:13:57 2022] POST /identity/v3/endpoints => generated 331 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:14:07.068160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=30 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:14:07.343920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:24.111799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ffee-43e1-4fe2-bef7-a4cd66b33774 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:24.112362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ffee-43e1-4fe2-bef7-a4cd66b33774 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:24.112362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb8ffee-43e1-4fe2-bef7-a4cd66b33774 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:24.114369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 643/1307] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:24.119665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e781c-02de-456f-9c1d-f877320af8dc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:24.120111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e781c-02de-456f-9c1d-f877320af8dc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:24.120329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8e781c-02de-456f-9c1d-f877320af8dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:24.235291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2a8e781c-02de-456f-9c1d-f877320af8dc None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:24.236557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2a8e781c-02de-456f-9c1d-f877320af8dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:24.466850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 665/1308] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:24.530041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5bc5f-ab13-4e76-90b9-3c6eb17f22a5 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:24.530041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5bc5f-ab13-4e76-90b9-3c6eb17f22a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:24.530041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c5bc5f-ab13-4e76-90b9-3c6eb17f22a5 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:24.530041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 644/1309] 10.222.0.70 () {54 vars in 964 bytes} [Mon May 23 13:22:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:22:24.537538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8d19-3b82-4fe3-a286-bc1f1644a699 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:24.537894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8d19-3b82-4fe3-a286-bc1f1644a699 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:24.538117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afad8d19-3b82-4fe3-a286-bc1f1644a699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:24.623174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-afad8d19-3b82-4fe3-a286-bc1f1644a699 None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:24.624420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-afad8d19-3b82-4fe3-a286-bc1f1644a699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:24.649178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:24.707043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 666/1310] 10.222.0.70 () {58 vars in 1061 bytes} [Mon May 23 13:22:24 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:24.716109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5bc5f-ab13-4e76-90b9-3c6eb17f22a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:24.718092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32c5bc5f-ab13-4e76-90b9-3c6eb17f22a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:24.848863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c5bc5f-ab13-4e76-90b9-3c6eb17f22a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:24.929151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:24.929811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:24.930096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:24.930376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:24.932038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:24.995025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:25.058692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:25.060154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364706e-fe64-4848-a838-cbb2e0012519 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:25.083012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 645/1311] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:27.598586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa8008-1cbf-4223-ab79-f3514dd6aefd None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:27.599107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa8008-1cbf-4223-ab79-f3514dd6aefd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:27.599225 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfa8008-1cbf-4223-ab79-f3514dd6aefd None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:27.600684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 667/1312] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:27.605927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc97c9c-8f4f-4f48-ab5b-cc34c6d1cf65 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:27.606398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc97c9c-8f4f-4f48-ab5b-cc34c6d1cf65 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:27.606490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc97c9c-8f4f-4f48-ab5b-cc34c6d1cf65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:27.686593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-dcc97c9c-8f4f-4f48-ab5b-cc34c6d1cf65 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:27.687820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dcc97c9c-8f4f-4f48-ab5b-cc34c6d1cf65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:27.789660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 646/1313] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:27.828148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa8008-1cbf-4223-ab79-f3514dd6aefd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:27.829817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3cfa8008-1cbf-4223-ab79-f3514dd6aefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:27.937197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfa8008-1cbf-4223-ab79-f3514dd6aefd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:28.009537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:28.012330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:28.013056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:28.013337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:28.013612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:28.015357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:28.130154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:28.131866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32632c7c-ffe9-4243-9451-5813f8a14ed1 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:28.152539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 668/1314] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:30.998080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1928-263e-4786-b7d2-9fea34e7d075 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:30.998689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1928-263e-4786-b7d2-9fea34e7d075 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:30.998689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5e1928-263e-4786-b7d2-9fea34e7d075 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:31.000146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 647/1315] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:31.006518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f426910-dc19-4db9-a27a-7d0ca813474c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:31.007040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f426910-dc19-4db9-a27a-7d0ca813474c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:31.007149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f426910-dc19-4db9-a27a-7d0ca813474c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:31.076819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7f426910-dc19-4db9-a27a-7d0ca813474c None None] MFA Rules not processed for user `d43e4a70df3c4919a2e14f9ad3265e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:31.077930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7f426910-dc19-4db9-a27a-7d0ca813474c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:31.130965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:31.183380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 669/1316] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:31 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:31.239723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07648f5e-7632-42ca-9eae-42bcdc87da31 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:31.240102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07648f5e-7632-42ca-9eae-42bcdc87da31 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:31.240451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07648f5e-7632-42ca-9eae-42bcdc87da31 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:31.241953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 648/1317] 10.222.0.70 () {54 vars in 964 bytes} [Mon May 23 13:22:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:31.248804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3d493-ef16-46a8-b195-1d3feb6c05e6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:31.254070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3d493-ef16-46a8-b195-1d3feb6c05e6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:31.254070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a3d493-ef16-46a8-b195-1d3feb6c05e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:31.326308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c3a3d493-ef16-46a8-b195-1d3feb6c05e6 None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:31.327572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c3a3d493-ef16-46a8-b195-1d3feb6c05e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:31.404088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 670/1318] 10.222.0.70 () {58 vars in 1061 bytes} [Mon May 23 13:22:31 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:31.413211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07648f5e-7632-42ca-9eae-42bcdc87da31 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:31.414761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07648f5e-7632-42ca-9eae-42bcdc87da31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:31.424423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:31.526060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07648f5e-7632-42ca-9eae-42bcdc87da31 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:31.601105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:31.601686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:31.601976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:31.602256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:31.604017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:31.667912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:31.715196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:31.716668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8691-aad7-425f-a197-8d39cf24853b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:31.736742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 649/1319] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:31 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:35.695919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c46b71-cc07-4292-9079-c7bb222548a8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:35.696451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c46b71-cc07-4292-9079-c7bb222548a8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:35.696548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c46b71-cc07-4292-9079-c7bb222548a8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:35.697933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 671/1320] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:35.703396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10588f2-d679-4a79-868b-5b1f6335c25f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:35.703814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10588f2-d679-4a79-868b-5b1f6335c25f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:35.703949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10588f2-d679-4a79-868b-5b1f6335c25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:35.769257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c10588f2-d679-4a79-868b-5b1f6335c25f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:35.770493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c10588f2-d679-4a79-868b-5b1f6335c25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:35.867416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 650/1321] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:35.906320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c46b71-cc07-4292-9079-c7bb222548a8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:35.907853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-79c46b71-cc07-4292-9079-c7bb222548a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:35.957343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:36.009848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c46b71-cc07-4292-9079-c7bb222548a8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:36.084778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:36.085441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:36.085731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:36.086006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:36.087696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:36.185841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:36.199464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:36.200931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcde2f9-96e2-43be-a795-e076700f2777 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:36.218955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 672/1322] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:39.905247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783ee6-f69c-4090-8ece-a8c0b569bc58 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:39.905819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783ee6-f69c-4090-8ece-a8c0b569bc58 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:39.905819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b783ee6-f69c-4090-8ece-a8c0b569bc58 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:39.907179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 651/1323] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:39.913095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ace3a-281c-45c6-b298-7e6a419e8751 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:39.913528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ace3a-281c-45c6-b298-7e6a419e8751 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:39.913658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ace3a-281c-45c6-b298-7e6a419e8751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:39.979476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-052ace3a-281c-45c6-b298-7e6a419e8751 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:39.980735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-052ace3a-281c-45c6-b298-7e6a419e8751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:40.078431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 673/1324] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:40.115395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783ee6-f69c-4090-8ece-a8c0b569bc58 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:40.116983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5b783ee6-f69c-4090-8ece-a8c0b569bc58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:40.136427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:40.223842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b783ee6-f69c-4090-8ece-a8c0b569bc58 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:40.306363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:40.307063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:40.307350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:40.307624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:40.309381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:40.384319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:40.423886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:40.426033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51356724-0758-4e1b-88a4-4be93ab6544b service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:40.445310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 652/1325] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:42.872352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7708927-5593-4cd2-8c78-f9ee37c6d07a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:42.872950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7708927-5593-4cd2-8c78-f9ee37c6d07a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:42.872950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7708927-5593-4cd2-8c78-f9ee37c6d07a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:42.874324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 674/1326] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:42.881526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7807a-0fd7-48e1-8bfb-9c8c7d0b06f1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:42.881948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7807a-0fd7-48e1-8bfb-9c8c7d0b06f1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:42.882097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e7807a-0fd7-48e1-8bfb-9c8c7d0b06f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:42.954177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e6e7807a-0fd7-48e1-8bfb-9c8c7d0b06f1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:42.956049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e6e7807a-0fd7-48e1-8bfb-9c8c7d0b06f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:43.056823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 653/1327] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:43.093280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7708927-5593-4cd2-8c78-f9ee37c6d07a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:43.094907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c7708927-5593-4cd2-8c78-f9ee37c6d07a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:43.156294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:43.200236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7708927-5593-4cd2-8c78-f9ee37c6d07a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:43.273776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:43.274435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:43.274708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:43.274984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:43.276657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:43.384213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:43.386559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:43.388005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683be4e8-9b2e-4c70-b8d9-19cecd4e853b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:43.405841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 675/1328] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:45.680268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405c1f-842e-4861-b528-7ba93dcd9384 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:45.680856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405c1f-842e-4861-b528-7ba93dcd9384 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:45.680856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d405c1f-842e-4861-b528-7ba93dcd9384 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:45.682228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 654/1329] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:45.687667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1d6cb-fe95-4995-87c3-f549d369b652 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:45.688090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1d6cb-fe95-4995-87c3-f549d369b652 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:45.688235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1d6cb-fe95-4995-87c3-f549d369b652 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:45.755731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-02b1d6cb-fe95-4995-87c3-f549d369b652 None None] MFA Rules not processed for user `d43e4a70df3c4919a2e14f9ad3265e10`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:45.757007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-02b1d6cb-fe95-4995-87c3-f549d369b652 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:45.853776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 676/1330] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:45 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:45.891425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405c1f-842e-4861-b528-7ba93dcd9384 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:45.893645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9d405c1f-842e-4861-b528-7ba93dcd9384 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:45.925905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:46.001606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d405c1f-842e-4861-b528-7ba93dcd9384 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:46.077012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:46.077668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:46.077948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:46.078218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:46.079936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:46.158512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:46.190963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:46.192427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1af3f95-ea23-40c8-9ea9-5a34062d9aae service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:46.211460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 655/1331] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:45 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:49.128283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33850c-ef7a-4e34-93d6-136286989a6d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:49.128829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33850c-ef7a-4e34-93d6-136286989a6d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:49.129011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea33850c-ef7a-4e34-93d6-136286989a6d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:49.130511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 677/1332] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:49.136005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a35b0-715e-462e-934a-6a3bf3255e06 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:49.136421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a35b0-715e-462e-934a-6a3bf3255e06 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:49.136595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a35b0-715e-462e-934a-6a3bf3255e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:49.206462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-db4a35b0-715e-462e-934a-6a3bf3255e06 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:49.207679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db4a35b0-715e-462e-934a-6a3bf3255e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:49.330854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 656/1333] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:49.367320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33850c-ef7a-4e34-93d6-136286989a6d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:49.368994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ea33850c-ef7a-4e34-93d6-136286989a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:49.444192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:49.479846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea33850c-ef7a-4e34-93d6-136286989a6d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:49.559074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:49.559725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:49.560003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:49.560274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:49.561948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:49.676202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:49.677778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996201c3-b3dc-49e4-980a-cc9b08f519a8 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:49.688532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:49.697423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 678/1334] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:51.974926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37cd7a-e4e5-49c5-8e5c-f5003deb0735 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:51.975493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37cd7a-e4e5-49c5-8e5c-f5003deb0735 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:51.975493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb37cd7a-e4e5-49c5-8e5c-f5003deb0735 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:51.976918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 657/1335] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:51.982457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4128fe9-4a59-4212-bef5-2c24620d37fd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:51.982864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4128fe9-4a59-4212-bef5-2c24620d37fd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:51.983014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4128fe9-4a59-4212-bef5-2c24620d37fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:52.050040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a4128fe9-4a59-4212-bef5-2c24620d37fd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:52.051285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a4128fe9-4a59-4212-bef5-2c24620d37fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:52.150071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 679/1336] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:22:52.190743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37cd7a-e4e5-49c5-8e5c-f5003deb0735 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:52.192304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb37cd7a-e4e5-49c5-8e5c-f5003deb0735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:52.234157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:52.303438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb37cd7a-e4e5-49c5-8e5c-f5003deb0735 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:52.382959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:52.383577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:52.383862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:52.384137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:52.385942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:52.481234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:52.508690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:52.510136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37caa44-45d3-4bf6-8026-1fa396139f04 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:52.529447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 658/1337] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:22:55.420009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd959f79-fd2a-48c7-92e5-caf44790cc57 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:55.420541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd959f79-fd2a-48c7-92e5-caf44790cc57 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:55.420740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd959f79-fd2a-48c7-92e5-caf44790cc57 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:55.421978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 680/1338] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:22:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:22:55.427901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b791-c918-4ef3-80c3-b9ff7053cf1c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:55.428350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b791-c918-4ef3-80c3-b9ff7053cf1c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:55.428470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a9b791-c918-4ef3-80c3-b9ff7053cf1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:55.498770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15a9b791-c918-4ef3-80c3-b9ff7053cf1c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:22:55.499991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15a9b791-c918-4ef3-80c3-b9ff7053cf1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:55.603203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 659/1339] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:22:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:22:55.639946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd959f79-fd2a-48c7-92e5-caf44790cc57 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:22:55.642202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cd959f79-fd2a-48c7-92e5-caf44790cc57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:22:55.726012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:22:55.744717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd959f79-fd2a-48c7-92e5-caf44790cc57 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:22:55.748356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b72533f-a7fd-4717-ab48-4d8df7ca84c3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:22:55.749107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72533f-a7fd-4717-ab48-4d8df7ca84c3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:22:55.749337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72533f-a7fd-4717-ab48-4d8df7ca84c3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:22:55.749617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b72533f-a7fd-4717-ab48-4d8df7ca84c3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:22:55.759499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72533f-a7fd-4717-ab48-4d8df7ca84c3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:22:55.760969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b72533f-a7fd-4717-ab48-4d8df7ca84c3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:22:55.770511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 681/1340] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:22:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:00.311836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad492f9-b6c1-4ec1-b2f5-c48fe0d88dc6 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:00.312390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad492f9-b6c1-4ec1-b2f5-c48fe0d88dc6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:00.312390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad492f9-b6c1-4ec1-b2f5-c48fe0d88dc6 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:00.313988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 660/1341] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:00.319454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cb873-36f4-458d-92ee-9b2f8f5b96a2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:00.319926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cb873-36f4-458d-92ee-9b2f8f5b96a2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:00.320047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781cb873-36f4-458d-92ee-9b2f8f5b96a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:00.350441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-781cb873-36f4-458d-92ee-9b2f8f5b96a2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:00.351632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-781cb873-36f4-458d-92ee-9b2f8f5b96a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:00.446157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 682/1342] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:23:00.484938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad492f9-b6c1-4ec1-b2f5-c48fe0d88dc6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:00.496867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad492f9-b6c1-4ec1-b2f5-c48fe0d88dc6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:00.500513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02adf4ec-0c0b-4dd4-8830-d596d29582bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:00.502313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adf4ec-0c0b-4dd4-8830-d596d29582bd service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:00.502601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adf4ec-0c0b-4dd4-8830-d596d29582bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:00.509226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02adf4ec-0c0b-4dd4-8830-d596d29582bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:00.512223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adf4ec-0c0b-4dd4-8830-d596d29582bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:00.513713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02adf4ec-0c0b-4dd4-8830-d596d29582bd service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:00.524187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 661/1343] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:03.918927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a665-8179-44a5-85f8-1ec73d2371d8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:03.919530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a665-8179-44a5-85f8-1ec73d2371d8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:03.919530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1a665-8179-44a5-85f8-1ec73d2371d8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:03.920878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 683/1344] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:03.926887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7febd-1f47-4452-b2a7-eafd4e892992 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:03.927334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7febd-1f47-4452-b2a7-eafd4e892992 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:03.927439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac7febd-1f47-4452-b2a7-eafd4e892992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:03.946086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ac7febd-1f47-4452-b2a7-eafd4e892992 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:03.947291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ac7febd-1f47-4452-b2a7-eafd4e892992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:03.955964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 662/1345] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:03 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:03.995204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a665-8179-44a5-85f8-1ec73d2371d8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:04.008129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc1a665-8179-44a5-85f8-1ec73d2371d8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:04.011765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cb38a0-0f29-4f6e-b4b7-38c30bd28f53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:04.012418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb38a0-0f29-4f6e-b4b7-38c30bd28f53 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:04.012749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb38a0-0f29-4f6e-b4b7-38c30bd28f53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:04.013029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cb38a0-0f29-4f6e-b4b7-38c30bd28f53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:04.024214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb38a0-0f29-4f6e-b4b7-38c30bd28f53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:04.025705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cb38a0-0f29-4f6e-b4b7-38c30bd28f53 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:04.036326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 684/1346] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:03 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:08.808722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23590cbe-d343-4d23-ab93-f9c3c89b2ee4 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:08.809229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23590cbe-d343-4d23-ab93-f9c3c89b2ee4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:08.809229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23590cbe-d343-4d23-ab93-f9c3c89b2ee4 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:08.810766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 663/1347] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:08.816222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183d3d3-57d2-43be-8e84-11a025b09255 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:08.816665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183d3d3-57d2-43be-8e84-11a025b09255 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:08.816821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8183d3d3-57d2-43be-8e84-11a025b09255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:08.835162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8183d3d3-57d2-43be-8e84-11a025b09255 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:08.836367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8183d3d3-57d2-43be-8e84-11a025b09255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:08.844280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 685/1348] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:08.883003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23590cbe-d343-4d23-ab93-f9c3c89b2ee4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:08.884956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-23590cbe-d343-4d23-ab93-f9c3c89b2ee4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:08.898773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23590cbe-d343-4d23-ab93-f9c3c89b2ee4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:08.902377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179ecaf1-1df8-47ad-9940-27b05ab59892 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:08.903025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ecaf1-1df8-47ad-9940-27b05ab59892 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:08.903310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ecaf1-1df8-47ad-9940-27b05ab59892 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:08.903639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179ecaf1-1df8-47ad-9940-27b05ab59892 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:08.913931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ecaf1-1df8-47ad-9940-27b05ab59892 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:08.915412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179ecaf1-1df8-47ad-9940-27b05ab59892 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:08.925170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 664/1349] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:14.659825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b62c-8c09-4da1-abfa-2bfff9fc0624 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:14.660697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b62c-8c09-4da1-abfa-2bfff9fc0624 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:14.661178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5793b62c-8c09-4da1-abfa-2bfff9fc0624 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:14.663007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 686/1350] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:23:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:14.668458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23cd22-7b28-42c2-a4f9-a71d76fd040f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:14.669270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23cd22-7b28-42c2-a4f9-a71d76fd040f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:14.669719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e23cd22-7b28-42c2-a4f9-a71d76fd040f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:14.691510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:14.753773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e23cd22-7b28-42c2-a4f9-a71d76fd040f None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:14.755311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e23cd22-7b28-42c2-a4f9-a71d76fd040f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:14.837437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 665/1351] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:23:14 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:18.829781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=57 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:19.049643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f06d4-773f-4b2b-bb2d-0c6836bdce1a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:19.049924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f06d4-773f-4b2b-bb2d-0c6836bdce1a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:19.050209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f06d4-773f-4b2b-bb2d-0c6836bdce1a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:19.051816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 687/1352] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:23:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:19.056731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e52a2-19eb-42ee-bafa-d4e6ba7bde56 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:19.057234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e52a2-19eb-42ee-bafa-d4e6ba7bde56 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:19.057558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-753e52a2-19eb-42ee-bafa-d4e6ba7bde56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:19.077896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-753e52a2-19eb-42ee-bafa-d4e6ba7bde56 None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:19.079261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-753e52a2-19eb-42ee-bafa-d4e6ba7bde56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:19.088443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 666/1353] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:23:19 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:20.828288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988898a7-640f-46b6-bc73-440f0c5a50a6 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:20.829052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988898a7-640f-46b6-bc73-440f0c5a50a6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:20.829052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988898a7-640f-46b6-bc73-440f0c5a50a6 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:20.830214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 688/1354] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:23:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:20.835175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0fa74-bd4b-4e51-bddc-55df41bfb49f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:20.835574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0fa74-bd4b-4e51-bddc-55df41bfb49f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:20.835874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0fa74-bd4b-4e51-bddc-55df41bfb49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:20.854892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d0a0fa74-bd4b-4e51-bddc-55df41bfb49f None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:20.856163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d0a0fa74-bd4b-4e51-bddc-55df41bfb49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:20.865227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 667/1355] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:23:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:24.851328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d429-0894-43de-bf7e-ec831f09f71d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:24.851897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d429-0894-43de-bf7e-ec831f09f71d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:24.851897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df0d429-0894-43de-bf7e-ec831f09f71d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:24.853279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 689/1356] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:24.859084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a992d4-6bf6-45c6-85de-13a81dcd5434 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:24.859929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a992d4-6bf6-45c6-85de-13a81dcd5434 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:24.860399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a992d4-6bf6-45c6-85de-13a81dcd5434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:24.879384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f0a992d4-6bf6-45c6-85de-13a81dcd5434 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:24.880946 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0a992d4-6bf6-45c6-85de-13a81dcd5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:24.889451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 668/1357] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:24.947323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c429a-351e-4d6f-8f3d-a5c94f30e045 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:24.948036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c429a-351e-4d6f-8f3d-a5c94f30e045 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:24.948769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146c429a-351e-4d6f-8f3d-a5c94f30e045 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:24.951218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 690/1358] 10.222.0.70 () {54 vars in 961 bytes} [Mon May 23 13:23:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:24.960403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1023045-a4a0-4176-8c31-d2305ef9fdea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:24.963965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1023045-a4a0-4176-8c31-d2305ef9fdea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:24.963965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1023045-a4a0-4176-8c31-d2305ef9fdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:25.023070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d1023045-a4a0-4176-8c31-d2305ef9fdea None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:25.024754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d1023045-a4a0-4176-8c31-d2305ef9fdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:25.099617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:25.123807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 669/1359] 10.222.0.70 () {58 vars in 1058 bytes} [Mon May 23 13:23:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:25.132255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c429a-351e-4d6f-8f3d-a5c94f30e045 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:25.148922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146c429a-351e-4d6f-8f3d-a5c94f30e045 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:25.154449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf22322-ffb2-4d38-98a1-2ee97e118736 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:25.155449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf22322-ffb2-4d38-98a1-2ee97e118736 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:25.155953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf22322-ffb2-4d38-98a1-2ee97e118736 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:25.156232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf22322-ffb2-4d38-98a1-2ee97e118736 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:25.166330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf22322-ffb2-4d38-98a1-2ee97e118736 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:25.168043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf22322-ffb2-4d38-98a1-2ee97e118736 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:25.178826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 691/1360] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:23:27.675609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e919ea8-0e8b-4c10-9207-95f5833b6621 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:27.676157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e919ea8-0e8b-4c10-9207-95f5833b6621 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:27.676157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e919ea8-0e8b-4c10-9207-95f5833b6621 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:27.677700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 670/1361] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:27.682819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0a8a9-8033-42ae-b684-de88d0be6e87 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:27.683214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0a8a9-8033-42ae-b684-de88d0be6e87 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:27.683376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd0a8a9-8033-42ae-b684-de88d0be6e87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:27.702503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ebd0a8a9-8033-42ae-b684-de88d0be6e87 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:27.704032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ebd0a8a9-8033-42ae-b684-de88d0be6e87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:27.719583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 692/1362] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:27.758457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e919ea8-0e8b-4c10-9207-95f5833b6621 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:27.773215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e919ea8-0e8b-4c10-9207-95f5833b6621 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:27.779096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e4720d-5349-40b1-ab86-4c0d4abefe01 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:27.779739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4720d-5349-40b1-ab86-4c0d4abefe01 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:27.780558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4720d-5349-40b1-ab86-4c0d4abefe01 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:27.780927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e4720d-5349-40b1-ab86-4c0d4abefe01 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:27.791959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4720d-5349-40b1-ab86-4c0d4abefe01 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:27.794159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e4720d-5349-40b1-ab86-4c0d4abefe01 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:27.805042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 671/1363] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:30.595364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7131d96-30bd-4678-b775-b91bba6a3cd8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:30.596355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7131d96-30bd-4678-b775-b91bba6a3cd8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:30.596355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7131d96-30bd-4678-b775-b91bba6a3cd8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:30.622489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 693/1364] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:30.622594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e179f7d-4789-47f9-ba75-89698d991712 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:30.622594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e179f7d-4789-47f9-ba75-89698d991712 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:30.622594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e179f7d-4789-47f9-ba75-89698d991712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:30.622594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5e179f7d-4789-47f9-ba75-89698d991712 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:30.624454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e179f7d-4789-47f9-ba75-89698d991712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:30.641859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 672/1365] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:23:30.692829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f7e9a-fbba-4c3a-9ecd-f8abe6dfdd02 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:30.693122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f7e9a-fbba-4c3a-9ecd-f8abe6dfdd02 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:30.693416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0f7e9a-fbba-4c3a-9ecd-f8abe6dfdd02 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:30.694852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 694/1366] 10.222.0.70 () {54 vars in 961 bytes} [Mon May 23 13:23:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:30.700935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e826bd-2a81-4e91-be4d-8f68ac39dac1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:30.701230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e826bd-2a81-4e91-be4d-8f68ac39dac1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:30.702082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e826bd-2a81-4e91-be4d-8f68ac39dac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:30.721640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b5e826bd-2a81-4e91-be4d-8f68ac39dac1 None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:30.722860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b5e826bd-2a81-4e91-be4d-8f68ac39dac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:30.733361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 673/1367] 10.222.0.70 () {58 vars in 1058 bytes} [Mon May 23 13:23:30 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:23:30.741086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f7e9a-fbba-4c3a-9ecd-f8abe6dfdd02 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:30.756902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0f7e9a-fbba-4c3a-9ecd-f8abe6dfdd02 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:30.762498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6fe4d4c-8bd7-4170-bb88-6c1c644a52c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:30.763139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe4d4c-8bd7-4170-bb88-6c1c644a52c9 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:30.772691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe4d4c-8bd7-4170-bb88-6c1c644a52c9 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:30.772691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fe4d4c-8bd7-4170-bb88-6c1c644a52c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:30.772964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe4d4c-8bd7-4170-bb88-6c1c644a52c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:30.774487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6fe4d4c-8bd7-4170-bb88-6c1c644a52c9 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:30.783942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 695/1368] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:23:33.330537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e589f5a-3db3-441c-abe3-7e672ccf309c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.331161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e589f5a-3db3-441c-abe3-7e672ccf309c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.331409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e589f5a-3db3-441c-abe3-7e672ccf309c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.333090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 674/1369] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:33.339056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a65d6-289d-4819-b120-03ad6759e9c6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.340509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a65d6-289d-4819-b120-03ad6759e9c6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.340509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3a65d6-289d-4819-b120-03ad6759e9c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.359971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ba3a65d6-289d-4819-b120-03ad6759e9c6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:33.361355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ba3a65d6-289d-4819-b120-03ad6759e9c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:33.370715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 696/1370] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:33.417262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e589f5a-3db3-441c-abe3-7e672ccf309c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:33.434384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e589f5a-3db3-441c-abe3-7e672ccf309c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:33.441021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade6fb12-efeb-4705-9016-13acf9ba8df4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:33.441698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6fb12-efeb-4705-9016-13acf9ba8df4 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.441987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6fb12-efeb-4705-9016-13acf9ba8df4 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.442269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade6fb12-efeb-4705-9016-13acf9ba8df4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.453799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6fb12-efeb-4705-9016-13acf9ba8df4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:33.455463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade6fb12-efeb-4705-9016-13acf9ba8df4 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:33.467397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 675/1371] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:33.501782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94a77e-42f5-4874-aacc-181eedb1ebdf None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.502075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94a77e-42f5-4874-aacc-181eedb1ebdf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.502788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a94a77e-42f5-4874-aacc-181eedb1ebdf None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.504351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 697/1372] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:33.510255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ef522-afc7-46db-9080-ccbfeb6904b5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.510552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ef522-afc7-46db-9080-ccbfeb6904b5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.511022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ef522-afc7-46db-9080-ccbfeb6904b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.531539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d39ef522-afc7-46db-9080-ccbfeb6904b5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:33.532837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d39ef522-afc7-46db-9080-ccbfeb6904b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:33.540857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 676/1373] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:33.818269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3367e11-8744-4ba9-802f-f3c11f1f1ba2 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.822709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3367e11-8744-4ba9-802f-f3c11f1f1ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.822709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3367e11-8744-4ba9-802f-f3c11f1f1ba2 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.822709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 698/1374] 10.222.0.70 () {54 vars in 963 bytes} [Mon May 23 13:23:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:33.825488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9d347-bf1c-4f6f-a2c7-a70df322a661 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.825916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9d347-bf1c-4f6f-a2c7-a70df322a661 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.826237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca9d347-bf1c-4f6f-a2c7-a70df322a661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:33.882516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7ca9d347-bf1c-4f6f-a2c7-a70df322a661 None None] MFA Rules not processed for user `1ff9d52fc1194828898369d1237b1337`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:33.884063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ca9d347-bf1c-4f6f-a2c7-a70df322a661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:33.965839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 677/1375] 10.222.0.70 () {58 vars in 1060 bytes} [Mon May 23 13:23:33 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:33.975030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3367e11-8744-4ba9-802f-f3c11f1f1ba2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:33.987729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3367e11-8744-4ba9-802f-f3c11f1f1ba2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:33.992234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92345769-8dd8-4903-8604-6dd07b83108e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:33.993174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92345769-8dd8-4903-8604-6dd07b83108e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:33.993753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92345769-8dd8-4903-8604-6dd07b83108e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:33.994347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92345769-8dd8-4903-8604-6dd07b83108e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:34.006563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92345769-8dd8-4903-8604-6dd07b83108e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:34.008486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92345769-8dd8-4903-8604-6dd07b83108e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:34.024262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 699/1376] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:36.137498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa72c8-9862-4e43-9c4c-176a62a320a3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:36.138050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa72c8-9862-4e43-9c4c-176a62a320a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:36.138162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fa72c8-9862-4e43-9c4c-176a62a320a3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:36.139491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 678/1377] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:36.145335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d8dac-aab7-460f-91b2-08957819aa4f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:36.145785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d8dac-aab7-460f-91b2-08957819aa4f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:36.145902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4d8dac-aab7-460f-91b2-08957819aa4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:36.167058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5b4d8dac-aab7-460f-91b2-08957819aa4f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:36.168281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5b4d8dac-aab7-460f-91b2-08957819aa4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:36.177092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 700/1378] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:23:36.223038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa72c8-9862-4e43-9c4c-176a62a320a3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:36.238929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fa72c8-9862-4e43-9c4c-176a62a320a3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:36.244372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e9580f-14c3-4ee3-9aae-e3378895fe67 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:36.245090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9580f-14c3-4ee3-9aae-e3378895fe67 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:36.245394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9580f-14c3-4ee3-9aae-e3378895fe67 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:36.245672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e9580f-14c3-4ee3-9aae-e3378895fe67 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:36.255599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9580f-14c3-4ee3-9aae-e3378895fe67 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:36.257135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e9580f-14c3-4ee3-9aae-e3378895fe67 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:36.267770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 679/1379] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:38.972055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb85c-6ca0-495a-8f1f-7d57ff5b6270 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:38.972863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb85c-6ca0-495a-8f1f-7d57ff5b6270 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:38.972863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e1eb85c-6ca0-495a-8f1f-7d57ff5b6270 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:38.974086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 701/1380] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:38.980119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14bfd-618f-4f01-a113-3ae8eeaaa7c9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:38.980815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14bfd-618f-4f01-a113-3ae8eeaaa7c9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:38.980815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb14bfd-618f-4f01-a113-3ae8eeaaa7c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:39.001666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1bb14bfd-618f-4f01-a113-3ae8eeaaa7c9 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:39.003183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1bb14bfd-618f-4f01-a113-3ae8eeaaa7c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:39.011590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 680/1381] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:39.056189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eb85c-6ca0-495a-8f1f-7d57ff5b6270 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:39.072771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e1eb85c-6ca0-495a-8f1f-7d57ff5b6270 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:39.080021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30089bb9-b6eb-40d3-bf83-96df8130ad97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:39.080711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089bb9-b6eb-40d3-bf83-96df8130ad97 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:39.081001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089bb9-b6eb-40d3-bf83-96df8130ad97 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:39.081282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30089bb9-b6eb-40d3-bf83-96df8130ad97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:39.090370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089bb9-b6eb-40d3-bf83-96df8130ad97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:39.092773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30089bb9-b6eb-40d3-bf83-96df8130ad97 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:39.103507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 702/1382] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:41.844460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a2c3-e52e-4cbd-9a5b-c2d11b1816c8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:41.845102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a2c3-e52e-4cbd-9a5b-c2d11b1816c8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:41.845102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b2a2c3-e52e-4cbd-9a5b-c2d11b1816c8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:41.846310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 681/1383] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:41.851961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ee14-6fa4-4ea3-9470-db072c70a40b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:41.852428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ee14-6fa4-4ea3-9470-db072c70a40b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:41.852669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf81ee14-6fa4-4ea3-9470-db072c70a40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:41.874044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bf81ee14-6fa4-4ea3-9470-db072c70a40b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:41.875494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bf81ee14-6fa4-4ea3-9470-db072c70a40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:41.883880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 703/1384] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:41.926579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a2c3-e52e-4cbd-9a5b-c2d11b1816c8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:41.942009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b2a2c3-e52e-4cbd-9a5b-c2d11b1816c8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:41.947528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a90ab3c-8a19-4373-8685-262be32e3e9f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:41.948174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90ab3c-8a19-4373-8685-262be32e3e9f service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:41.948461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90ab3c-8a19-4373-8685-262be32e3e9f service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:41.948764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a90ab3c-8a19-4373-8685-262be32e3e9f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:41.958142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90ab3c-8a19-4373-8685-262be32e3e9f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:41.959755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a90ab3c-8a19-4373-8685-262be32e3e9f service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:41.971626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 682/1385] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:44.450241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9261a-c582-48e5-9076-ad9a5b44736a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:44.450904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9261a-c582-48e5-9076-ad9a5b44736a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:44.450904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a9261a-c582-48e5-9076-ad9a5b44736a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:44.452369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 704/1386] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:44.458046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb2f41-2ae7-49c5-8e64-4a44deb95a5e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:44.458645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb2f41-2ae7-49c5-8e64-4a44deb95a5e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:44.458645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fb2f41-2ae7-49c5-8e64-4a44deb95a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:44.477607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-25fb2f41-2ae7-49c5-8e64-4a44deb95a5e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:44.478814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-25fb2f41-2ae7-49c5-8e64-4a44deb95a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:44.488018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 683/1387] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:44.533250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9261a-c582-48e5-9076-ad9a5b44736a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:44.548341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a9261a-c582-48e5-9076-ad9a5b44736a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:44.554919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9898449-6591-4c51-818b-3522aae2d901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:44.555652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9898449-6591-4c51-818b-3522aae2d901 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:44.555945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9898449-6591-4c51-818b-3522aae2d901 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:44.556234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9898449-6591-4c51-818b-3522aae2d901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:44.566026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9898449-6591-4c51-818b-3522aae2d901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:44.567633 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9898449-6591-4c51-818b-3522aae2d901 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:44.577869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 705/1388] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:47.071761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa336e5-a023-4f08-928c-6b72819e648a None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:47.072654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa336e5-a023-4f08-928c-6b72819e648a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:47.073110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa336e5-a023-4f08-928c-6b72819e648a None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:47.078871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 684/1389] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:47.081117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9274d-9003-43e3-826d-14e0ba17a986 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:47.081984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9274d-9003-43e3-826d-14e0ba17a986 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:47.082222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9274d-9003-43e3-826d-14e0ba17a986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:47.103386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cff9274d-9003-43e3-826d-14e0ba17a986 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:47.104670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cff9274d-9003-43e3-826d-14e0ba17a986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:47.120256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 706/1390] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:47.164597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa336e5-a023-4f08-928c-6b72819e648a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:47.182396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa336e5-a023-4f08-928c-6b72819e648a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:47.191346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b077de-e1d3-4520-b9ad-5aee5b3ec985 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:47.192294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b077de-e1d3-4520-b9ad-5aee5b3ec985 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:47.192758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b077de-e1d3-4520-b9ad-5aee5b3ec985 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:47.192993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b077de-e1d3-4520-b9ad-5aee5b3ec985 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:47.202512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b077de-e1d3-4520-b9ad-5aee5b3ec985 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:47.204272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b077de-e1d3-4520-b9ad-5aee5b3ec985 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:47.217691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 685/1391] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:49.850917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259f586-bea6-4f66-b44b-3f9d995382c4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:49.851522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259f586-bea6-4f66-b44b-3f9d995382c4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:49.851522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9259f586-bea6-4f66-b44b-3f9d995382c4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:49.853041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 707/1392] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:49.859199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da457ccc-e7f9-4203-9c5e-2b4a10ba077c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:49.859643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da457ccc-e7f9-4203-9c5e-2b4a10ba077c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:49.859838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da457ccc-e7f9-4203-9c5e-2b4a10ba077c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:49.878595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da457ccc-e7f9-4203-9c5e-2b4a10ba077c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:49.879814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da457ccc-e7f9-4203-9c5e-2b4a10ba077c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:49.888556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 686/1393] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:49.933904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259f586-bea6-4f66-b44b-3f9d995382c4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:49.942047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=46 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:49.948639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9259f586-bea6-4f66-b44b-3f9d995382c4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:49.955160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6693b6a5-00e2-439d-9afe-ef339a806dd6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:49.960308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693b6a5-00e2-439d-9afe-ef339a806dd6 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:49.960308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693b6a5-00e2-439d-9afe-ef339a806dd6 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:49.960308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6693b6a5-00e2-439d-9afe-ef339a806dd6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:49.964667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693b6a5-00e2-439d-9afe-ef339a806dd6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:49.967309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6693b6a5-00e2-439d-9afe-ef339a806dd6 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:49.977824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 708/1394] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:50.371932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93eb2-ec03-4e8d-9aa3-6157b8642f61 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:50.372858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93eb2-ec03-4e8d-9aa3-6157b8642f61 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:50.373015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da93eb2-ec03-4e8d-9aa3-6157b8642f61 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:50.374406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 687/1395] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:50.380886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbaa0c-0417-4c9a-a33d-06e1c474e535 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:50.381184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbaa0c-0417-4c9a-a33d-06e1c474e535 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:50.381485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbaa0c-0417-4c9a-a33d-06e1c474e535 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:50.401774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-60bbaa0c-0417-4c9a-a33d-06e1c474e535 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:50.403680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-60bbaa0c-0417-4c9a-a33d-06e1c474e535 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:50.415814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 709/1396] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:50.475911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25fc1-ffba-4c64-a755-b60ef0a99f1d None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:50.476285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25fc1-ffba-4c64-a755-b60ef0a99f1d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:50.476768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d25fc1-ffba-4c64-a755-b60ef0a99f1d None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:50.478332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 688/1397] 10.222.0.70 () {54 vars in 962 bytes} [Mon May 23 13:23:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:50.490700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4b3d5-c860-4186-8d6b-671491a2879f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:50.490988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4b3d5-c860-4186-8d6b-671491a2879f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:50.491272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c4b3d5-c860-4186-8d6b-671491a2879f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:50.542628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a6c4b3d5-c860-4186-8d6b-671491a2879f None None] MFA Rules not processed for user `99a35674d90447d0a7ee255845280be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:50.543820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a6c4b3d5-c860-4186-8d6b-671491a2879f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:50.606197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 710/1398] 10.222.0.70 () {58 vars in 1059 bytes} [Mon May 23 13:23:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:50.614704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25fc1-ffba-4c64-a755-b60ef0a99f1d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:50.625996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d25fc1-ffba-4c64-a755-b60ef0a99f1d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:50.629667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4328a69-600a-4376-af22-48371d603893 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:50.630334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328a69-600a-4376-af22-48371d603893 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:50.630621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328a69-600a-4376-af22-48371d603893 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:50.630898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4328a69-600a-4376-af22-48371d603893 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:50.641211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4328a69-600a-4376-af22-48371d603893 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:50.648096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4328a69-600a-4376-af22-48371d603893 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:50.652459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 689/1399] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:51.422845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b20b-3122-42fb-9a76-fce3049dd2e7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:51.423399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b20b-3122-42fb-9a76-fce3049dd2e7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:51.423557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d95b20b-3122-42fb-9a76-fce3049dd2e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:51.472427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8d95b20b-3122-42fb-9a76-fce3049dd2e7 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:51.473674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8d95b20b-3122-42fb-9a76-fce3049dd2e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:51.539615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 711/1400] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:23:51 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:51.584843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5927215-785b-4100-8ff8-876980d76624 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:51.585311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5927215-785b-4100-8ff8-876980d76624 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:51.585428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5927215-785b-4100-8ff8-876980d76624 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:51.586831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 690/1401] 10.222.0.70 () {54 vars in 942 bytes} [Mon May 23 13:23:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:51.594151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556fa0f-0f49-4da8-a0a9-29c3b0d53598 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:51.594586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556fa0f-0f49-4da8-a0a9-29c3b0d53598 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:51.594969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d556fa0f-0f49-4da8-a0a9-29c3b0d53598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:51.641894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:51.645107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d556fa0f-0f49-4da8-a0a9-29c3b0d53598 None None] MFA Rules not processed for user `e525e14157f94459a6e2ab46ffdeb529`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:51.646520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d556fa0f-0f49-4da8-a0a9-29c3b0d53598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:51.718343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 712/1402] 10.222.0.70 () {58 vars in 1039 bytes} [Mon May 23 13:23:51 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:51.728017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5927215-785b-4100-8ff8-876980d76624 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:51.742065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5927215-785b-4100-8ff8-876980d76624 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:51.746961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff69757d-2403-479c-8b34-ece40773fdc6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:51.747602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69757d-2403-479c-8b34-ece40773fdc6 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:51.747906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69757d-2403-479c-8b34-ece40773fdc6 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:51.748165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff69757d-2403-479c-8b34-ece40773fdc6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:51.753491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:23:51.757364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69757d-2403-479c-8b34-ece40773fdc6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:51.759026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff69757d-2403-479c-8b34-ece40773fdc6 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:51.767103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 691/1403] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:23:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:23:52.503207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea00167-b6ba-409c-a95e-e94ea1d2eaa7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:52.503714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea00167-b6ba-409c-a95e-e94ea1d2eaa7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:52.503714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea00167-b6ba-409c-a95e-e94ea1d2eaa7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:52.505085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 713/1404] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:52.510972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e4e4f-c63c-4951-b934-be43e8bbec7a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:52.511268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e4e4f-c63c-4951-b934-be43e8bbec7a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:52.511555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0e4e4f-c63c-4951-b934-be43e8bbec7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:52.546060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0f0e4e4f-c63c-4951-b934-be43e8bbec7a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:52.547328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f0e4e4f-c63c-4951-b934-be43e8bbec7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:52.557559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 692/1405] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:23:52.603130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea00167-b6ba-409c-a95e-e94ea1d2eaa7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:52.618593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea00167-b6ba-409c-a95e-e94ea1d2eaa7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:52.624396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ea6596-d1f5-405a-a009-3976cd445718 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:52.625077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea6596-d1f5-405a-a009-3976cd445718 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:52.625362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea6596-d1f5-405a-a009-3976cd445718 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:52.625645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ea6596-d1f5-405a-a009-3976cd445718 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:52.635794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea6596-d1f5-405a-a009-3976cd445718 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:52.637481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ea6596-d1f5-405a-a009-3976cd445718 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:52.647874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 714/1406] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:23:55.258389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999940f-e93b-47cf-a0df-7cbcbd62bc07 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:55.258944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999940f-e93b-47cf-a0df-7cbcbd62bc07 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:55.258944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e999940f-e93b-47cf-a0df-7cbcbd62bc07 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:55.260421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 693/1407] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:23:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:23:55.266188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd21f3c-d6e2-4e8a-8fa5-2883b8623bab None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:55.266790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd21f3c-d6e2-4e8a-8fa5-2883b8623bab None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:55.266790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd21f3c-d6e2-4e8a-8fa5-2883b8623bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:55.284973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6bd21f3c-d6e2-4e8a-8fa5-2883b8623bab None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:23:55.286208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6bd21f3c-d6e2-4e8a-8fa5-2883b8623bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:23:55.294924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 715/1408] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:23:55.340010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999940f-e93b-47cf-a0df-7cbcbd62bc07 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:23:55.355525 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e999940f-e93b-47cf-a0df-7cbcbd62bc07 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:23:55.362523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dd3a62-a4cf-4061-988e-3562e35e605e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:23:55.363167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd3a62-a4cf-4061-988e-3562e35e605e service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:23:55.363472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd3a62-a4cf-4061-988e-3562e35e605e service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:23:55.363749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dd3a62-a4cf-4061-988e-3562e35e605e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:23:55.373953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd3a62-a4cf-4061-988e-3562e35e605e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:23:55.376875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dd3a62-a4cf-4061-988e-3562e35e605e service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:23:55.387398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 694/1409] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:05.443989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=39 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:05.445102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:17.545874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa7fa-4e78-4e95-9bfa-6040c7280c62 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:17.546433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa7fa-4e78-4e95-9bfa-6040c7280c62 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:17.546433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4fa7fa-4e78-4e95-9bfa-6040c7280c62 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:17.547742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 716/1410] 10.222.0.70 () {54 vars in 918 bytes} [Mon May 23 13:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:17.553214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca06aa8-e72d-4046-91fb-98f8f9036c34 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:17.553643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca06aa8-e72d-4046-91fb-98f8f9036c34 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:17.553810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca06aa8-e72d-4046-91fb-98f8f9036c34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:17.572183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aca06aa8-e72d-4046-91fb-98f8f9036c34 None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:17.573926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aca06aa8-e72d-4046-91fb-98f8f9036c34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:17.582664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 695/1411] 10.222.0.70 () {58 vars in 1015 bytes} [Mon May 23 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:17.597439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e223cf-e7a5-4bb2-b6a1-813d5d26264a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:17.597734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e223cf-e7a5-4bb2-b6a1-813d5d26264a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:17.598020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e223cf-e7a5-4bb2-b6a1-813d5d26264a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:17.599392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 717/1412] 10.222.0.70 () {54 vars in 918 bytes} [Mon May 23 13:24:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:17.604215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826db1e-decf-4fc9-8d9c-368af006cf0f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:17.605171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826db1e-decf-4fc9-8d9c-368af006cf0f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:17.605431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826db1e-decf-4fc9-8d9c-368af006cf0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:17.623179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0826db1e-decf-4fc9-8d9c-368af006cf0f None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:17.624372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0826db1e-decf-4fc9-8d9c-368af006cf0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:17.634204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 696/1413] 10.222.0.70 () {58 vars in 1015 bytes} [Mon May 23 13:24:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:20.338954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8370c7-1395-4c02-ac92-19785e45434f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:20.339496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8370c7-1395-4c02-ac92-19785e45434f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:20.339601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8370c7-1395-4c02-ac92-19785e45434f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:20.341026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 718/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:20.346328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6a1ac-3177-4775-9833-1b8cf3ec8482 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:20.346747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6a1ac-3177-4775-9833-1b8cf3ec8482 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:20.346896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6a1ac-3177-4775-9833-1b8cf3ec8482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:20.364459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b8a6a1ac-3177-4775-9833-1b8cf3ec8482 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:20.365680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8a6a1ac-3177-4775-9833-1b8cf3ec8482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:20.374285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 697/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:20.416521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8370c7-1395-4c02-ac92-19785e45434f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:20.430694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8370c7-1395-4c02-ac92-19785e45434f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:20.437474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fefcbb-a850-4868-b827-7ec1ce582497 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:20.437658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fefcbb-a850-4868-b827-7ec1ce582497 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:20.437860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fefcbb-a850-4868-b827-7ec1ce582497 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:20.438132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fefcbb-a850-4868-b827-7ec1ce582497 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:20.447216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fefcbb-a850-4868-b827-7ec1ce582497 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:20.449433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8fefcbb-a850-4868-b827-7ec1ce582497 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:20.459268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 719/1416] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:21.436134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41fad-01d3-4c6d-a837-228aa9170bf3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:21.436675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41fad-01d3-4c6d-a837-228aa9170bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:21.436675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e41fad-01d3-4c6d-a837-228aa9170bf3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:21.438084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 698/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:21.443153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963998-e414-4f0b-938b-9686a1ce6471 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:21.443557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963998-e414-4f0b-938b-9686a1ce6471 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:21.443715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36963998-e414-4f0b-938b-9686a1ce6471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:21.470640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-36963998-e414-4f0b-938b-9686a1ce6471 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:21.472598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-36963998-e414-4f0b-938b-9686a1ce6471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:21.485179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 720/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:21.648840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41fad-01d3-4c6d-a837-228aa9170bf3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:21.663829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e41fad-01d3-4c6d-a837-228aa9170bf3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:21.669539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8255c8e3-b48c-4da6-a0c8-b09acb25d0c2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:21.670166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255c8e3-b48c-4da6-a0c8-b09acb25d0c2 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:21.670446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255c8e3-b48c-4da6-a0c8-b09acb25d0c2 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:21.670773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8255c8e3-b48c-4da6-a0c8-b09acb25d0c2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:21.679387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255c8e3-b48c-4da6-a0c8-b09acb25d0c2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:21.680973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8255c8e3-b48c-4da6-a0c8-b09acb25d0c2 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:21.691868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 699/1419] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:22.640409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae1b9e-c8c6-46d3-a6a0-2d2561612fff None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:22.640990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae1b9e-c8c6-46d3-a6a0-2d2561612fff None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:22.640990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ae1b9e-c8c6-46d3-a6a0-2d2561612fff None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:22.642376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 721/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:22.647344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457457d0-e8a3-4eb5-ab6d-5c3099ac2417 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:22.647769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457457d0-e8a3-4eb5-ab6d-5c3099ac2417 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:22.647908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457457d0-e8a3-4eb5-ab6d-5c3099ac2417 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:22.665748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-457457d0-e8a3-4eb5-ab6d-5c3099ac2417 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:22.667295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-457457d0-e8a3-4eb5-ab6d-5c3099ac2417 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:22.675513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 700/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:22.832098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae1b9e-c8c6-46d3-a6a0-2d2561612fff None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:22.853426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97ae1b9e-c8c6-46d3-a6a0-2d2561612fff None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:22.859738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52b1778-88d1-4c95-96da-0a98aa7372b3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:22.860378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1778-88d1-4c95-96da-0a98aa7372b3 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:22.860683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1778-88d1-4c95-96da-0a98aa7372b3 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:22.860956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52b1778-88d1-4c95-96da-0a98aa7372b3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:22.869680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b1778-88d1-4c95-96da-0a98aa7372b3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:22.871922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52b1778-88d1-4c95-96da-0a98aa7372b3 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:22.881674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 722/1422] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:23.809834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38f5cf-6c59-49c2-b08d-efd658d43325 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:23.810393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38f5cf-6c59-49c2-b08d-efd658d43325 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:23.810393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db38f5cf-6c59-49c2-b08d-efd658d43325 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:23.811804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 701/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:23.816880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251523e-b18c-4c67-ae3c-5f52bb37be3f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:23.817316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251523e-b18c-4c67-ae3c-5f52bb37be3f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:23.817445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6251523e-b18c-4c67-ae3c-5f52bb37be3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:23.835598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6251523e-b18c-4c67-ae3c-5f52bb37be3f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:23.836838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6251523e-b18c-4c67-ae3c-5f52bb37be3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:23.845112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 723/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:23.996560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38f5cf-6c59-49c2-b08d-efd658d43325 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:24.012087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db38f5cf-6c59-49c2-b08d-efd658d43325 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:24.017805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cefa9b8-2ce6-4b7f-a2fe-88a420e2cc6c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:24.018452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefa9b8-2ce6-4b7f-a2fe-88a420e2cc6c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:24.018733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefa9b8-2ce6-4b7f-a2fe-88a420e2cc6c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:24.019006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cefa9b8-2ce6-4b7f-a2fe-88a420e2cc6c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:24.028099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefa9b8-2ce6-4b7f-a2fe-88a420e2cc6c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:24.029727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cefa9b8-2ce6-4b7f-a2fe-88a420e2cc6c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:24.039187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 702/1425] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:24.983067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437b75a-5cfa-43fc-86f1-2d2bd8d72111 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:24.983606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437b75a-5cfa-43fc-86f1-2d2bd8d72111 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:24.983606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5437b75a-5cfa-43fc-86f1-2d2bd8d72111 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:24.985041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 724/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:24.990168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d5adf-9a3f-46f8-86e4-fe2afe311e0a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:24.990613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d5adf-9a3f-46f8-86e4-fe2afe311e0a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:24.990774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d5adf-9a3f-46f8-86e4-fe2afe311e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:25.009966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-254d5adf-9a3f-46f8-86e4-fe2afe311e0a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:25.011776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-254d5adf-9a3f-46f8-86e4-fe2afe311e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:25.020121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 703/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:25.169690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437b75a-5cfa-43fc-86f1-2d2bd8d72111 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:25.184757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5437b75a-5cfa-43fc-86f1-2d2bd8d72111 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:25.190114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89105d5f-b6e7-47cb-a9b1-66679d1df993 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:25.190762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89105d5f-b6e7-47cb-a9b1-66679d1df993 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:25.191047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89105d5f-b6e7-47cb-a9b1-66679d1df993 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:25.191326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89105d5f-b6e7-47cb-a9b1-66679d1df993 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:25.200280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89105d5f-b6e7-47cb-a9b1-66679d1df993 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:25.201895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89105d5f-b6e7-47cb-a9b1-66679d1df993 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:25.212541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 725/1428] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:26.159555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197e5a-6fab-4f4d-8e27-1c5572af7309 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:26.160125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197e5a-6fab-4f4d-8e27-1c5572af7309 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:26.160125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44197e5a-6fab-4f4d-8e27-1c5572af7309 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:26.161583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 704/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:26.166776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4bb0-7d41-4e5b-976f-690accff634e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:26.167212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4bb0-7d41-4e5b-976f-690accff634e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:26.167345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee4bb0-7d41-4e5b-976f-690accff634e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:26.185396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-23ee4bb0-7d41-4e5b-976f-690accff634e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:26.186616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23ee4bb0-7d41-4e5b-976f-690accff634e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:26.194450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 726/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:26.348357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197e5a-6fab-4f4d-8e27-1c5572af7309 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:26.362681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44197e5a-6fab-4f4d-8e27-1c5572af7309 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:26.368060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd0b57b-1322-4bd9-80ae-66ec0789afcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:26.368805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0b57b-1322-4bd9-80ae-66ec0789afcc service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:26.369092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0b57b-1322-4bd9-80ae-66ec0789afcc service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:26.369899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd0b57b-1322-4bd9-80ae-66ec0789afcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:26.378809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0b57b-1322-4bd9-80ae-66ec0789afcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:26.380415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd0b57b-1322-4bd9-80ae-66ec0789afcc service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:26.390497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 705/1431] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:27.361045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0eea7-7d9b-408e-ab2e-9a5455e6c6b1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:27.361605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0eea7-7d9b-408e-ab2e-9a5455e6c6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:27.361605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d0eea7-7d9b-408e-ab2e-9a5455e6c6b1 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:27.363049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 727/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:27.367918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379245a9-0830-48b2-b30d-f32355949377 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:27.368398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379245a9-0830-48b2-b30d-f32355949377 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:27.368504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-379245a9-0830-48b2-b30d-f32355949377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:27.387733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-379245a9-0830-48b2-b30d-f32355949377 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:27.389021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-379245a9-0830-48b2-b30d-f32355949377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:27.396900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 706/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:27.550813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0eea7-7d9b-408e-ab2e-9a5455e6c6b1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:27.565953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d0eea7-7d9b-408e-ab2e-9a5455e6c6b1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:27.571431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac2564d-6dee-4c16-8db0-967d5f5bad34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:27.572090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2564d-6dee-4c16-8db0-967d5f5bad34 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:27.573010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2564d-6dee-4c16-8db0-967d5f5bad34 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:27.573298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac2564d-6dee-4c16-8db0-967d5f5bad34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:27.582059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2564d-6dee-4c16-8db0-967d5f5bad34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:27.591232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac2564d-6dee-4c16-8db0-967d5f5bad34 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:27.593240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 728/1434] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:28.562235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef75501-c8ea-4499-91b0-a93906774efa None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:28.562726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef75501-c8ea-4499-91b0-a93906774efa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:28.562839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef75501-c8ea-4499-91b0-a93906774efa None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:28.564183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 707/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:28.569237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c3a8e-f123-4ac8-95b5-7cf9070c6181 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:28.569635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c3a8e-f123-4ac8-95b5-7cf9070c6181 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:28.569794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c3a8e-f123-4ac8-95b5-7cf9070c6181 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:28.588551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ec3c3a8e-f123-4ac8-95b5-7cf9070c6181 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:28.589755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ec3c3a8e-f123-4ac8-95b5-7cf9070c6181 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:28.597474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 729/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:28.747224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef75501-c8ea-4499-91b0-a93906774efa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:28.762941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef75501-c8ea-4499-91b0-a93906774efa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:28.774001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ddd7570-9a5d-40d6-9d02-6016334331c1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:28.774660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd7570-9a5d-40d6-9d02-6016334331c1 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:28.774950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd7570-9a5d-40d6-9d02-6016334331c1 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:28.775266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ddd7570-9a5d-40d6-9d02-6016334331c1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:28.783985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd7570-9a5d-40d6-9d02-6016334331c1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:28.785667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ddd7570-9a5d-40d6-9d02-6016334331c1 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:28.795398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 708/1437] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:29.739326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc49aa-6f50-45d7-ba38-06bf1d7796e8 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:29.739868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc49aa-6f50-45d7-ba38-06bf1d7796e8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:29.739868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bc49aa-6f50-45d7-ba38-06bf1d7796e8 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:29.741346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 730/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:24:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:29.746361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b203-3c7c-4a44-b10a-68703f0e363b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:29.746806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b203-3c7c-4a44-b10a-68703f0e363b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:29.746953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72b203-3c7c-4a44-b10a-68703f0e363b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:29.773837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1b72b203-3c7c-4a44-b10a-68703f0e363b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:29.775045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1b72b203-3c7c-4a44-b10a-68703f0e363b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:29.783849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 709/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:24:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:29.943616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc49aa-6f50-45d7-ba38-06bf1d7796e8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:29.959008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bc49aa-6f50-45d7-ba38-06bf1d7796e8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:29.965201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6adc53-106f-4e10-9628-e1bf29b849fa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:29.965839 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6adc53-106f-4e10-9628-e1bf29b849fa service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:29.966123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6adc53-106f-4e10-9628-e1bf29b849fa service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:29.966404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6adc53-106f-4e10-9628-e1bf29b849fa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:29.974391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6adc53-106f-4e10-9628-e1bf29b849fa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:29.975958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6adc53-106f-4e10-9628-e1bf29b849fa service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:29.985248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 731/1440] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:32.079640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5769d8-80f3-4b71-85e3-286ada5af1b3 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:32.080224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5769d8-80f3-4b71-85e3-286ada5af1b3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:32.080224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5769d8-80f3-4b71-85e3-286ada5af1b3 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:32.081583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 710/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:32.087140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c607b96-00aa-4c56-8c8a-c2ac216e1066 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:32.087663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c607b96-00aa-4c56-8c8a-c2ac216e1066 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:32.088390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c607b96-00aa-4c56-8c8a-c2ac216e1066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:32.105750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8c607b96-00aa-4c56-8c8a-c2ac216e1066 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:32.106946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8c607b96-00aa-4c56-8c8a-c2ac216e1066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:32.114565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 732/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:32.154107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5769d8-80f3-4b71-85e3-286ada5af1b3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:32.164736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5769d8-80f3-4b71-85e3-286ada5af1b3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:32.168257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c815a37-719a-4d64-ab23-82dea8496da8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:32.168916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c815a37-719a-4d64-ab23-82dea8496da8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:32.169200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c815a37-719a-4d64-ab23-82dea8496da8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:32.169477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c815a37-719a-4d64-ab23-82dea8496da8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:32.178914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c815a37-719a-4d64-ab23-82dea8496da8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:32.180441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c815a37-719a-4d64-ab23-82dea8496da8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:32.190568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 711/1443] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:34.575597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a84a3-f0cf-49fd-895d-b3a1f5763cc7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:34.576133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a84a3-f0cf-49fd-895d-b3a1f5763cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:34.576133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318a84a3-f0cf-49fd-895d-b3a1f5763cc7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:34.577496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 733/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:24:34.582991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98b22d-8ef5-4727-8973-27eaa9737a7e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:34.583444 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98b22d-8ef5-4727-8973-27eaa9737a7e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:34.583551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f98b22d-8ef5-4727-8973-27eaa9737a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:34.601626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7f98b22d-8ef5-4727-8973-27eaa9737a7e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:34.602810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7f98b22d-8ef5-4727-8973-27eaa9737a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:34.610847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 712/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:34.650456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a84a3-f0cf-49fd-895d-b3a1f5763cc7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:34.661146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318a84a3-f0cf-49fd-895d-b3a1f5763cc7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:34.664714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309baefb-65e2-4963-80ad-2482efa2b588 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:34.665353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309baefb-65e2-4963-80ad-2482efa2b588 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:34.665694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309baefb-65e2-4963-80ad-2482efa2b588 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:34.665984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309baefb-65e2-4963-80ad-2482efa2b588 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:34.674751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309baefb-65e2-4963-80ad-2482efa2b588 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:34.676175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309baefb-65e2-4963-80ad-2482efa2b588 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:34.686025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 734/1446] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:37.447563 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca19b5-6a13-42d2-b5d0-3d557be27dbd None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:37.448134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca19b5-6a13-42d2-b5d0-3d557be27dbd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:37.448134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ca19b5-6a13-42d2-b5d0-3d557be27dbd None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:37.449464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 713/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:37.455117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8833c-dbed-4b5f-a9d8-13ed15cb7d67 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:37.455560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8833c-dbed-4b5f-a9d8-13ed15cb7d67 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:37.455670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f8833c-dbed-4b5f-a9d8-13ed15cb7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:37.473021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c5f8833c-dbed-4b5f-a9d8-13ed15cb7d67 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:37.474222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5f8833c-dbed-4b5f-a9d8-13ed15cb7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:37.482160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 735/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:37.521738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca19b5-6a13-42d2-b5d0-3d557be27dbd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:37.532730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ca19b5-6a13-42d2-b5d0-3d557be27dbd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:37.536312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b537c8-3180-44d7-bd9a-b38d62dbf106 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:37.537007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b537c8-3180-44d7-bd9a-b38d62dbf106 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:37.537295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b537c8-3180-44d7-bd9a-b38d62dbf106 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:37.537569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b537c8-3180-44d7-bd9a-b38d62dbf106 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:37.554229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b537c8-3180-44d7-bd9a-b38d62dbf106 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:37.555734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b537c8-3180-44d7-bd9a-b38d62dbf106 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:37.572581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 714/1449] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:39.951299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50eebd7-7dbb-48d6-9d0f-a8cbe62d144d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:39.951865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50eebd7-7dbb-48d6-9d0f-a8cbe62d144d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:39.951865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50eebd7-7dbb-48d6-9d0f-a8cbe62d144d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:39.953275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 736/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:39.958644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5281-39e4-4be5-a5bc-a3d40090446a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:39.959080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5281-39e4-4be5-a5bc-a3d40090446a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:39.959214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371d5281-39e4-4be5-a5bc-a3d40090446a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:39.978149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-371d5281-39e4-4be5-a5bc-a3d40090446a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:39.979421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-371d5281-39e4-4be5-a5bc-a3d40090446a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:39.987306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 715/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:40.029005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50eebd7-7dbb-48d6-9d0f-a8cbe62d144d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:40.039610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50eebd7-7dbb-48d6-9d0f-a8cbe62d144d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:40.043396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1bf9a1-ec23-4e6b-9213-70aeae5a55c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:40.044027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bf9a1-ec23-4e6b-9213-70aeae5a55c4 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:40.044308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bf9a1-ec23-4e6b-9213-70aeae5a55c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:40.044640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1bf9a1-ec23-4e6b-9213-70aeae5a55c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:40.054028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bf9a1-ec23-4e6b-9213-70aeae5a55c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:40.055466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1bf9a1-ec23-4e6b-9213-70aeae5a55c4 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:40.065622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 737/1452] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:24:42.413397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9589a-22ba-4c0e-8b25-ded7619dc21c None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:42.413968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9589a-22ba-4c0e-8b25-ded7619dc21c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:42.413968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e9589a-22ba-4c0e-8b25-ded7619dc21c None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:42.415565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 716/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:42.421304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b405-a8e4-4a11-877f-1df89484072f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:42.421840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b405-a8e4-4a11-877f-1df89484072f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:42.421948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e545b405-a8e4-4a11-877f-1df89484072f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:42.440506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e545b405-a8e4-4a11-877f-1df89484072f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:42.441687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e545b405-a8e4-4a11-877f-1df89484072f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:42.449898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 738/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:42.486934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9589a-22ba-4c0e-8b25-ded7619dc21c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:42.498094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e9589a-22ba-4c0e-8b25-ded7619dc21c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:42.501815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c81779-8d13-4223-8732-abe0662a34d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:42.502442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c81779-8d13-4223-8732-abe0662a34d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:42.502725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c81779-8d13-4223-8732-abe0662a34d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:42.502999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c81779-8d13-4223-8732-abe0662a34d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:42.512708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c81779-8d13-4223-8732-abe0662a34d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:42.514160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c81779-8d13-4223-8732-abe0662a34d7 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:42.523712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 717/1455] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:44.889798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76081b-45d0-4463-ad86-7b956eaa16f4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:44.890354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76081b-45d0-4463-ad86-7b956eaa16f4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:44.890354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f76081b-45d0-4463-ad86-7b956eaa16f4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:44.891739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 739/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:44.897338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170a205-cc79-44dd-9b9b-8ee073ec84aa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:44.897781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170a205-cc79-44dd-9b9b-8ee073ec84aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:44.897880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b170a205-cc79-44dd-9b9b-8ee073ec84aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:44.917255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b170a205-cc79-44dd-9b9b-8ee073ec84aa None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:44.918523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b170a205-cc79-44dd-9b9b-8ee073ec84aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:44.926447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 718/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:24:44.965063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76081b-45d0-4463-ad86-7b956eaa16f4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:44.975778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f76081b-45d0-4463-ad86-7b956eaa16f4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:44.979869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209ce1a0-f08e-4f81-8385-a285cbd9bcda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:44.980506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ce1a0-f08e-4f81-8385-a285cbd9bcda service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:44.980820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ce1a0-f08e-4f81-8385-a285cbd9bcda service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:44.981101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209ce1a0-f08e-4f81-8385-a285cbd9bcda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:44.989967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ce1a0-f08e-4f81-8385-a285cbd9bcda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:44.991397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209ce1a0-f08e-4f81-8385-a285cbd9bcda service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:45.002012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 740/1458] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:47.329584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4130-a130-428d-a643-45c326320773 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:47.330124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4130-a130-428d-a643-45c326320773 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:47.330263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-099d4130-a130-428d-a643-45c326320773 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:47.331557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 719/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:47.337372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef676722-c5c7-43af-8183-84df1ec09614 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:47.337823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef676722-c5c7-43af-8183-84df1ec09614 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:47.337930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef676722-c5c7-43af-8183-84df1ec09614 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:47.344138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=70 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:47.356462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ef676722-c5c7-43af-8183-84df1ec09614 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:47.359427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ef676722-c5c7-43af-8183-84df1ec09614 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:47.365955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 741/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:47.403861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4130-a130-428d-a643-45c326320773 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:47.409229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=73 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:24:47.415364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-099d4130-a130-428d-a643-45c326320773 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:47.422377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cd8661-a469-448c-89a9-a27b13a40e1f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:47.422377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8661-a469-448c-89a9-a27b13a40e1f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:47.422377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8661-a469-448c-89a9-a27b13a40e1f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:47.422377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cd8661-a469-448c-89a9-a27b13a40e1f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:47.428890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd8661-a469-448c-89a9-a27b13a40e1f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:47.430951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cd8661-a469-448c-89a9-a27b13a40e1f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:47.440745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 720/1461] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:49.776449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19217e-5e1d-4aa9-9d57-3e616a092589 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:49.777048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19217e-5e1d-4aa9-9d57-3e616a092589 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:49.777048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19217e-5e1d-4aa9-9d57-3e616a092589 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:49.778432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 742/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:49.783616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f578b-8e54-42c9-aba9-b943dcbb6fca None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:49.784064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f578b-8e54-42c9-aba9-b943dcbb6fca None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:49.784169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41f578b-8e54-42c9-aba9-b943dcbb6fca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:49.803118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a41f578b-8e54-42c9-aba9-b943dcbb6fca None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:49.804299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a41f578b-8e54-42c9-aba9-b943dcbb6fca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:49.812765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 721/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:49.849962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19217e-5e1d-4aa9-9d57-3e616a092589 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:49.861704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19217e-5e1d-4aa9-9d57-3e616a092589 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:49.865247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf48c73-a595-4bda-b9b6-87f649bc1fd2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:49.865888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf48c73-a595-4bda-b9b6-87f649bc1fd2 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:49.866178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf48c73-a595-4bda-b9b6-87f649bc1fd2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:49.866471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf48c73-a595-4bda-b9b6-87f649bc1fd2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:49.876630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf48c73-a595-4bda-b9b6-87f649bc1fd2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:49.878312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf48c73-a595-4bda-b9b6-87f649bc1fd2 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:49.891973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 743/1464] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:52.161774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5044c4-1cc9-43da-b991-4eb1e64fca32 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:52.162342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5044c4-1cc9-43da-b991-4eb1e64fca32 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:52.162342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df5044c4-1cc9-43da-b991-4eb1e64fca32 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:52.163709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 722/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:52.169920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7f4d5-5c80-4ee2-aa9e-f763ff0dfbb7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:52.177206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7f4d5-5c80-4ee2-aa9e-f763ff0dfbb7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:52.177206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a7f4d5-5c80-4ee2-aa9e-f763ff0dfbb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:52.189233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-96a7f4d5-5c80-4ee2-aa9e-f763ff0dfbb7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:52.190449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-96a7f4d5-5c80-4ee2-aa9e-f763ff0dfbb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:52.198883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 744/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:52.239297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5044c4-1cc9-43da-b991-4eb1e64fca32 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:52.252882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df5044c4-1cc9-43da-b991-4eb1e64fca32 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:52.256693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d94a04d-830b-456c-bfcb-703c8a0f22c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:52.257338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a04d-830b-456c-bfcb-703c8a0f22c2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:52.257628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a04d-830b-456c-bfcb-703c8a0f22c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:52.258100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d94a04d-830b-456c-bfcb-703c8a0f22c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:52.267956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94a04d-830b-456c-bfcb-703c8a0f22c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:52.269465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d94a04d-830b-456c-bfcb-703c8a0f22c2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:52.279629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 723/1467] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:24:54.546520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2aff-4f1a-474e-b214-7bdec735632c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:54.547851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2aff-4f1a-474e-b214-7bdec735632c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:54.547851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ace2aff-4f1a-474e-b214-7bdec735632c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:54.548484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 745/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:24:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:24:54.553735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f2c25-bfd0-41bc-a99c-0b94bb78c64a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:54.554140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f2c25-bfd0-41bc-a99c-0b94bb78c64a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:54.554284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f2c25-bfd0-41bc-a99c-0b94bb78c64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:54.573303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-662f2c25-bfd0-41bc-a99c-0b94bb78c64a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:24:54.574494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-662f2c25-bfd0-41bc-a99c-0b94bb78c64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:24:54.582367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 724/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:24:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:24:54.619018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2aff-4f1a-474e-b214-7bdec735632c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:24:54.630032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ace2aff-4f1a-474e-b214-7bdec735632c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:24:54.634222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c2856dc-6aea-4502-9f2d-cb3b573e35dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:24:54.634873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2856dc-6aea-4502-9f2d-cb3b573e35dc service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:24:54.635158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2856dc-6aea-4502-9f2d-cb3b573e35dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:24:54.635433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2856dc-6aea-4502-9f2d-cb3b573e35dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:24:54.645062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2856dc-6aea-4502-9f2d-cb3b573e35dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:24:54.646528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c2856dc-6aea-4502-9f2d-cb3b573e35dc service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:24:54.655836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 746/1470] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:24:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:25:04.567651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:25:04.651845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=17 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:25:29.088036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172dc60-e760-4965-ba8a-7a613f7100a2 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:29.088674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172dc60-e760-4965-ba8a-7a613f7100a2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:29.088674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e172dc60-e760-4965-ba8a-7a613f7100a2 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:29.089980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 725/1471] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:29.095163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde452cf-ffca-4b45-8e76-ce5933cac689 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:29.095599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde452cf-ffca-4b45-8e76-ce5933cac689 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:29.095728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde452cf-ffca-4b45-8e76-ce5933cac689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:29.114256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dde452cf-ffca-4b45-8e76-ce5933cac689 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:29.116006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dde452cf-ffca-4b45-8e76-ce5933cac689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:29.124164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 747/1472] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:25:29.171174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00541d-e88a-478f-8fb9-9c0c269ee854 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:29.171462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00541d-e88a-478f-8fb9-9c0c269ee854 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:29.171742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00541d-e88a-478f-8fb9-9c0c269ee854 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:29.173172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 726/1473] 10.222.0.70 () {54 vars in 962 bytes} [Mon May 23 13:25:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:29.179665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcc991-97ba-4713-91c3-2f2ef7628c23 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:29.181989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcc991-97ba-4713-91c3-2f2ef7628c23 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:29.181989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebcc991-97ba-4713-91c3-2f2ef7628c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:29.198392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aebcc991-97ba-4713-91c3-2f2ef7628c23 None None] MFA Rules not processed for user `99a35674d90447d0a7ee255845280be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:29.199644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aebcc991-97ba-4713-91c3-2f2ef7628c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:29.206138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 748/1474] 10.222.0.70 () {58 vars in 1059 bytes} [Mon May 23 13:25:29 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:25:29.214941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00541d-e88a-478f-8fb9-9c0c269ee854 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:29.227925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da00541d-e88a-478f-8fb9-9c0c269ee854 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:29.231599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7bd05-aec1-4873-933b-65357ca92be6 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:29.232299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7bd05-aec1-4873-933b-65357ca92be6 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:29.232745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7bd05-aec1-4873-933b-65357ca92be6 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:29.233034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7bd05-aec1-4873-933b-65357ca92be6 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:29.243621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7bd05-aec1-4873-933b-65357ca92be6 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:29.245194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7bd05-aec1-4873-933b-65357ca92be6 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:29.254991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 727/1475] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:25:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:25:31.626772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1dcf4-ed21-48f9-94ad-3a2ad8a1501a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:31.627345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1dcf4-ed21-48f9-94ad-3a2ad8a1501a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:31.627345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf1dcf4-ed21-48f9-94ad-3a2ad8a1501a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:31.628780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 749/1476] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:31.634619 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e71449-4074-42bc-a35b-ac0ec875ad53 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:31.635008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e71449-4074-42bc-a35b-ac0ec875ad53 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:31.635173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e71449-4074-42bc-a35b-ac0ec875ad53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:31.653442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4e71449-4074-42bc-a35b-ac0ec875ad53 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:31.654637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4e71449-4074-42bc-a35b-ac0ec875ad53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:31.662538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 728/1477] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:25:31.685784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1dcf4-ed21-48f9-94ad-3a2ad8a1501a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:31.699024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf1dcf4-ed21-48f9-94ad-3a2ad8a1501a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:31.703397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:31.704037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:31.704317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:31.704613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:31.705347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:31.707181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:25:31.707181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:25:31.707181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:25:31.708936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d443ee2-6360-47b9-977e-4c42eb56443b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:31.729011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 750/1478] 10.222.0.70 () {56 vars in 1108 bytes} [Mon May 23 13:25:31 2022] GET /identity/v3/projects => generated 2553 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:25:33.968637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a8446-3786-4d6f-86a3-0b377e86c298 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:33.969179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a8446-3786-4d6f-86a3-0b377e86c298 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:33.969179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926a8446-3786-4d6f-86a3-0b377e86c298 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:33.970577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 729/1479] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:33.976540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434e484-9d20-428f-8f97-5a26b42ada3f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:33.977012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434e484-9d20-428f-8f97-5a26b42ada3f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:33.977157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7434e484-9d20-428f-8f97-5a26b42ada3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:33.996263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7434e484-9d20-428f-8f97-5a26b42ada3f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:33.997515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7434e484-9d20-428f-8f97-5a26b42ada3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:34.005595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 751/1480] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:25:34.052503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a8446-3786-4d6f-86a3-0b377e86c298 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:34.067281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926a8446-3786-4d6f-86a3-0b377e86c298 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:34.073240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508c20d2-684e-40fd-aacc-499840933951 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:34.073885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c20d2-684e-40fd-aacc-499840933951 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:34.074170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c20d2-684e-40fd-aacc-499840933951 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:34.074449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508c20d2-684e-40fd-aacc-499840933951 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:34.083598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c20d2-684e-40fd-aacc-499840933951 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:34.085383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508c20d2-684e-40fd-aacc-499840933951 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:34.094801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 730/1481] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:25:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:25:36.493581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357532-732d-4b5a-86d3-3ee243669024 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:36.494109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357532-732d-4b5a-86d3-3ee243669024 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:36.494109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3357532-732d-4b5a-86d3-3ee243669024 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:36.495756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 752/1482] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:36.502388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c668-3817-4934-9596-d5e592e72e22 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:36.502388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c668-3817-4934-9596-d5e592e72e22 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:36.502388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f8c668-3817-4934-9596-d5e592e72e22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:36.520155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-26f8c668-3817-4934-9596-d5e592e72e22 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:36.522040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26f8c668-3817-4934-9596-d5e592e72e22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:36.529838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 731/1483] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:25:36.565216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357532-732d-4b5a-86d3-3ee243669024 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:36.576662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3357532-732d-4b5a-86d3-3ee243669024 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:36.580134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f33db7-7f95-49ce-9c73-b11b00dde276 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:36.580852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33db7-7f95-49ce-9c73-b11b00dde276 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:36.581183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33db7-7f95-49ce-9c73-b11b00dde276 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:36.581460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f33db7-7f95-49ce-9c73-b11b00dde276 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:36.591035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f33db7-7f95-49ce-9c73-b11b00dde276 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:36.592451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f33db7-7f95-49ce-9c73-b11b00dde276 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:36.603355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 753/1484] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:25:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:25:38.857236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b0e5e-63e8-47cb-81d5-602cf84b4736 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:38.857880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b0e5e-63e8-47cb-81d5-602cf84b4736 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:38.857880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d41b0e5e-63e8-47cb-81d5-602cf84b4736 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:38.859181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 732/1485] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:38.864872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0869ff-effe-4199-839b-9a0d5f98dfb2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:38.865320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0869ff-effe-4199-839b-9a0d5f98dfb2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:38.865422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa0869ff-effe-4199-839b-9a0d5f98dfb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:38.883121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-aa0869ff-effe-4199-839b-9a0d5f98dfb2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:38.884312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aa0869ff-effe-4199-839b-9a0d5f98dfb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:38.891973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 754/1486] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:25:38.929481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b0e5e-63e8-47cb-81d5-602cf84b4736 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:38.940083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d41b0e5e-63e8-47cb-81d5-602cf84b4736 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:38.943759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ef1e59-e334-455c-98ec-17bf37d54b0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:38.944381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1e59-e334-455c-98ec-17bf37d54b0d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:38.944689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1e59-e334-455c-98ec-17bf37d54b0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:38.944966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ef1e59-e334-455c-98ec-17bf37d54b0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:38.953734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef1e59-e334-455c-98ec-17bf37d54b0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:38.955436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ef1e59-e334-455c-98ec-17bf37d54b0d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:38.965568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 733/1487] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:25:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:25:41.381132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378bed2-984b-4be8-a74d-1302a6b70cc3 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:41.381706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378bed2-984b-4be8-a74d-1302a6b70cc3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:41.381706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f378bed2-984b-4be8-a74d-1302a6b70cc3 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:41.383072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 755/1488] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:41.389150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3768b5-61db-4ad2-8141-22558ae65b58 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:41.389597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3768b5-61db-4ad2-8141-22558ae65b58 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:41.389741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3768b5-61db-4ad2-8141-22558ae65b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:41.407747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5b3768b5-61db-4ad2-8141-22558ae65b58 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:41.408990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5b3768b5-61db-4ad2-8141-22558ae65b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:41.417339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 734/1489] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:25:41.458480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378bed2-984b-4be8-a74d-1302a6b70cc3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:41.469027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f378bed2-984b-4be8-a74d-1302a6b70cc3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:41.472442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a680b5c-2024-4341-995b-4630fcbf6b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:41.473690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680b5c-2024-4341-995b-4630fcbf6b78 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:41.473987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680b5c-2024-4341-995b-4630fcbf6b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:41.474261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a680b5c-2024-4341-995b-4630fcbf6b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:41.483072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680b5c-2024-4341-995b-4630fcbf6b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:41.484495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a680b5c-2024-4341-995b-4630fcbf6b78 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:41.494267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 756/1490] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:25:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:25:44.162987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772961dc-30e7-4ffa-9973-3508b82caf38 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:44.163547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772961dc-30e7-4ffa-9973-3508b82caf38 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:44.163547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772961dc-30e7-4ffa-9973-3508b82caf38 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:44.165023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 735/1491] 10.222.0.70 () {54 vars in 926 bytes} [Mon May 23 13:25:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:25:44.170445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85dc34-6700-4e2a-8fd3-411833bd3dc5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:44.170874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85dc34-6700-4e2a-8fd3-411833bd3dc5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:44.171012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85dc34-6700-4e2a-8fd3-411833bd3dc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:44.190168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bb85dc34-6700-4e2a-8fd3-411833bd3dc5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:25:44.191416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bb85dc34-6700-4e2a-8fd3-411833bd3dc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:25:44.204965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 757/1492] 10.222.0.70 () {58 vars in 1023 bytes} [Mon May 23 13:25:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:25:44.245209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772961dc-30e7-4ffa-9973-3508b82caf38 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:25:44.256206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772961dc-30e7-4ffa-9973-3508b82caf38 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:25:44.259788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5b2c2-9fa3-4440-a284-1672d530c720 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:25:44.260484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5b2c2-9fa3-4440-a284-1672d530c720 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:25:44.260800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5b2c2-9fa3-4440-a284-1672d530c720 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:25:44.261075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5b2c2-9fa3-4440-a284-1672d530c720 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:25:44.270326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5b2c2-9fa3-4440-a284-1672d530c720 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:25:44.271826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5b2c2-9fa3-4440-a284-1672d530c720 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:25:44.281469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 736/1493] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:25:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:25:54.184869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:25:54.277044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:27:39.471731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:39.472306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:39.472306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:39.492437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:27:39.493669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:39.501972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 758/1494] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:27:39.511713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:39.512246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:39.512345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:39.534544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:27:39.535767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:39.544434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 737/1495] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:27:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:27:39.549075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:39.559383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51edf756-1e76-4eae-83bd-4880386ba261 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:39.563871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:39.564528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:39.564836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:39.565111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:39.565844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:27:39.566152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:39.567169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:39.567169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:27:39.567169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:27:39.568093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:39.579477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 759/1496] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:27:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:27:39.587736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:39.597329 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48437b3d-f275-4b79-b485-99455e4696f7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:39.601989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:39.602671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:39.602954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:39.603230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:39.603960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:39.606447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:39.606447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:27:39.606447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:27:39.607682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:39.720102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 738/1497] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:27:39 2022] POST /identity/v3/projects => generated 375 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:27:39.724496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:39.735623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dcbaaa7-2d24-43c2-9d98-f40172f19370 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:39.793277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:39.793955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:39.794245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:39.794571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:39.795238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:39.796254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:39.796254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:27:39.796254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:27:39.797504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:39.958455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 760/1498] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:27:39 2022] POST /identity/v3/users => generated 288 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:27:39.962584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:39.974001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babb49f4-b7b7-4bd0-a231-f681bd75a584 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:39.978639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:39.979275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:39.980815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:39.981133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:39.981873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:39.982914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:39.982914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:27:39.982914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:27:39.983800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:39.997975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 739/1499] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:27:39 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:27:40.002053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:40.012253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95937035-7a5c-4328-93f7-ee948f8f207b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:40.016841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:40.017535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.017820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.018099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.018829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:40.019868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:40.019868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:27:40.019868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:27:40.020779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f43345-78b7-4180-a481-0f045327a31a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:40.035357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 761/1500] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:27:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:27:40.039736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:40.051083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9757196e-59dd-49a2-a883-7decd03902ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:40.055705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:40.056610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.056904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.057184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] PATH_INFO: `/v3/projects/20e8cdd6717d4eb9974304e566368286/users/854bd8b60c504df9b6224e2b700756d4/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.072337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20e8cdd6717d4eb9974304e566368286, user_id=854bd8b60c504df9b6224e2b700756d4, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:40.074039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:40.074039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:27:40.074039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:27:40.075994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df1c7a25-78b6-43ce-9e01-767badc25d95 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:40.153512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 740/1501] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:27:40 2022] PUT /identity/v3/projects/20e8cdd6717d4eb9974304e566368286/users/854bd8b60c504df9b6224e2b700756d4/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:27:40.164680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9694f-c3db-4b20-a0aa-536f96c42592 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.165000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9694f-c3db-4b20-a0aa-536f96c42592 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.165281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e9694f-c3db-4b20-a0aa-536f96c42592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.193395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-03e9694f-c3db-4b20-a0aa-536f96c42592 None None] MFA Rules not processed for user `854bd8b60c504df9b6224e2b700756d4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:27:40.194611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-03e9694f-c3db-4b20-a0aa-536f96c42592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:40.258978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 762/1502] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:27:40.281762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafcd1d-e0d3-422e-9b42-2b6b203c0f45 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.282057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafcd1d-e0d3-422e-9b42-2b6b203c0f45 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.282344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafcd1d-e0d3-422e-9b42-2b6b203c0f45 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.283741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 741/1503] 10.222.0.70 () {54 vars in 963 bytes} [Mon May 23 13:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:27:40.288668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efa283-8276-42f8-89b4-52af41022083 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.288962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efa283-8276-42f8-89b4-52af41022083 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.289245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44efa283-8276-42f8-89b4-52af41022083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.307311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-44efa283-8276-42f8-89b4-52af41022083 None None] MFA Rules not processed for user `1ff9d52fc1194828898369d1237b1337`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:27:40.308707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-44efa283-8276-42f8-89b4-52af41022083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:40.352067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:27:40.359872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 763/1504] 10.222.0.70 () {58 vars in 1060 bytes} [Mon May 23 13:27:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:27:40.473043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafcd1d-e0d3-422e-9b42-2b6b203c0f45 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:40.483654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafcd1d-e0d3-422e-9b42-2b6b203c0f45 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:40.487183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:40.488201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.488489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.493676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.497683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:40.499155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:40.508467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 742/1505] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:27:40 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:27:40.553692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886c1f3-10d8-400b-a0dc-ef8264bda564 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.554169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886c1f3-10d8-400b-a0dc-ef8264bda564 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.554282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0886c1f3-10d8-400b-a0dc-ef8264bda564 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.555595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 764/1506] 10.222.0.70 () {52 vars in 829 bytes} [Mon May 23 13:27:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:27:40.559975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:40.653786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081fb612-fc08-4c40-b9eb-9ad58c47b426 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:40.658607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:40.659487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:40.659768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:40.660055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] PATH_INFO: `/v3/users/854bd8b60c504df9b6224e2b700756d4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:40.661275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=854bd8b60c504df9b6224e2b700756d4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:40.662306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:40.662306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:27:40.662306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:27:40.663498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:40.954116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '854bd8b60c504df9b6224e2b700756d4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:40.968331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '854bd8b60c504df9b6224e2b700756d4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:40.989418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '854bd8b60c504df9b6224e2b700756d4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:40.990282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '854bd8b60c504df9b6224e2b700756d4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:40.994324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:27:41.455220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '854bd8b60c504df9b6224e2b700756d4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:41.462032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 743/1507] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:27:40 2022] DELETE /identity/v3/users/854bd8b60c504df9b6224e2b700756d4 => generated 0 bytes in 903 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:27:41.466785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:41.467078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:41.467358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:41.485194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:27:41.485863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:41.546328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 765/1508] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:27:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:27:41.556161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:41.559207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:41.617148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e27f55d2-6ecb-41db-a7a5-9844a7dc6fa2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:41.621430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5b8a9-fdea-4877-b09a-6ccb9085e3ad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:41.622156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5b8a9-fdea-4877-b09a-6ccb9085e3ad service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:41.622442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5b8a9-fdea-4877-b09a-6ccb9085e3ad service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:41.622719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5b8a9-fdea-4877-b09a-6ccb9085e3ad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:41.631736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5b8a9-fdea-4877-b09a-6ccb9085e3ad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:41.633220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5b8a9-fdea-4877-b09a-6ccb9085e3ad service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:41.643379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 744/1509] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:27:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:27:42.239958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:27:42.241481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:27:42.253950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2285f400-6afb-47ef-84c0-b12393ee5304 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:27:42.261605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:27:42.262642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:27:42.263088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:27:42.263490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] PATH_INFO: `/v3/projects/20e8cdd6717d4eb9974304e566368286` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:27:42.265575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20e8cdd6717d4eb9974304e566368286)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:27:42.266852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:27:42.266852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:27:42.266852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:27:42.268080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:27:42.415504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20e8cdd6717d4eb9974304e566368286', 'request_id': 'req-91459290-e252-4d63-924c-f22b71508503'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:42.429342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20e8cdd6717d4eb9974304e566368286', 'request_id': 'req-91459290-e252-4d63-924c-f22b71508503'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:42.436086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20e8cdd6717d4eb9974304e566368286', 'request_id': 'req-91459290-e252-4d63-924c-f22b71508503'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:42.546646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] The token cache is being invalidate because project 20e8cdd6717d4eb9974304e566368286 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:27:42.546870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:42.547465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-91459290-e252-4d63-924c-f22b71508503 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:27:42.549191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 766/1510] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:27:42 2022] DELETE /identity/v3/projects/20e8cdd6717d4eb9974304e566368286 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:27:51.639146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:27:52.543232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:29:25.493968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d33478-9646-4eda-b9c3-f483b4686089 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:25.494598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d33478-9646-4eda-b9c3-f483b4686089 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:25.494598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d33478-9646-4eda-b9c3-f483b4686089 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:25.495832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 745/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:29:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:25.501093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f25de9-69aa-4d86-acd2-0d7fe11819c5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:25.501636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f25de9-69aa-4d86-acd2-0d7fe11819c5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:25.501636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f25de9-69aa-4d86-acd2-0d7fe11819c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:25.522923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c1f25de9-69aa-4d86-acd2-0d7fe11819c5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:25.524165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c1f25de9-69aa-4d86-acd2-0d7fe11819c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:25.593196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 767/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:29:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:29:25.773643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d33478-9646-4eda-b9c3-f483b4686089 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:25.775377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b4d33478-9646-4eda-b9c3-f483b4686089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:25.864357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d33478-9646-4eda-b9c3-f483b4686089 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:25.871288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61128405-7c9b-4e62-a319-7c0b9037cbb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:25.871950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61128405-7c9b-4e62-a319-7c0b9037cbb5 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:25.872240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61128405-7c9b-4e62-a319-7c0b9037cbb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:25.872522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61128405-7c9b-4e62-a319-7c0b9037cbb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:25.881934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61128405-7c9b-4e62-a319-7c0b9037cbb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:25.883565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61128405-7c9b-4e62-a319-7c0b9037cbb5 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:25.893748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 746/1513] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:26.810074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3befa-204e-450a-99da-23afc560e926 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:26.810607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3befa-204e-450a-99da-23afc560e926 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:26.810607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d3befa-204e-450a-99da-23afc560e926 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:26.811910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 768/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:29:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:26.816788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:26.817260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:26.817400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:26.817513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b82d38-d3f8-4c42-881e-29c75042b3b1 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:26.818416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b82d38-d3f8-4c42-881e-29c75042b3b1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:26.818701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b82d38-d3f8-4c42-881e-29c75042b3b1 None None] PATH_INFO: `/v3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:26.819978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 769/1515] 10.222.0.67 () {54 vars in 925 bytes} [Mon May 23 13:29:26 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:29:26.826116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056c71-1992-44e7-a1d3-28e417a8052e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:26.826401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056c71-1992-44e7-a1d3-28e417a8052e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:26.826700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe056c71-1992-44e7-a1d3-28e417a8052e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:26.843998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:26.845292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:26.848363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fe056c71-1992-44e7-a1d3-28e417a8052e None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:26.849640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fe056c71-1992-44e7-a1d3-28e417a8052e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:26.857783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 747/1516] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:29:26.917243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 770/1517] 10.222.0.67 () {58 vars in 1015 bytes} [Mon May 23 13:29:26 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:29:27.012897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:27.026254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f54e9da-2fbd-41bc-825f-4247439bcb32 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:27.034180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f6b326-8a90-49e1-93c4-799b145bb23a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:27.034881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b326-8a90-49e1-93c4-799b145bb23a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:27.036030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b326-8a90-49e1-93c4-799b145bb23a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:27.036030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f6b326-8a90-49e1-93c4-799b145bb23a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:27.045338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6b326-8a90-49e1-93c4-799b145bb23a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:27.047016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9f6b326-8a90-49e1-93c4-799b145bb23a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:27.058524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 748/1518] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:36.333315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e342b0-6b0f-496d-ac7d-8b438100c90c None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:36.334073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e342b0-6b0f-496d-ac7d-8b438100c90c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:36.334073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e342b0-6b0f-496d-ac7d-8b438100c90c None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:36.335934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 771/1519] 10.222.0.70 () {54 vars in 960 bytes} [Mon May 23 13:29:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:36.341883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69150c-53a9-45c4-8400-4278a64a1853 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:36.342499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69150c-53a9-45c4-8400-4278a64a1853 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:36.342499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e69150c-53a9-45c4-8400-4278a64a1853 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:36.363652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5e69150c-53a9-45c4-8400-4278a64a1853 None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:36.365801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5e69150c-53a9-45c4-8400-4278a64a1853 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:36.374885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 749/1520] 10.222.0.70 () {58 vars in 1057 bytes} [Mon May 23 13:29:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:29:36.383541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e342b0-6b0f-496d-ac7d-8b438100c90c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:36.398074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e342b0-6b0f-496d-ac7d-8b438100c90c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:36.403721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cfd5ad-4538-4329-b506-f2d326ab6222 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:36.404462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd5ad-4538-4329-b506-f2d326ab6222 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:36.404859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd5ad-4538-4329-b506-f2d326ab6222 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:36.405262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cfd5ad-4538-4329-b506-f2d326ab6222 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:36.415431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd5ad-4538-4329-b506-f2d326ab6222 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:36.417209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90cfd5ad-4538-4329-b506-f2d326ab6222 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:36.432848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 772/1521] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:36 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:38.484418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca822e3-3490-427d-83ce-814fd7933a63 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:38.485117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca822e3-3490-427d-83ce-814fd7933a63 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:38.485117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca822e3-3490-427d-83ce-814fd7933a63 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:38.486434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 750/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:29:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:38.492442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7b0a0-2f1e-4303-85fb-baccec5deebd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:38.493277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7b0a0-2f1e-4303-85fb-baccec5deebd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:38.493443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e7b0a0-2f1e-4303-85fb-baccec5deebd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:38.514745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f4e7b0a0-2f1e-4303-85fb-baccec5deebd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:38.516913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f4e7b0a0-2f1e-4303-85fb-baccec5deebd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:38.530951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 773/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:29:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:29:38.696694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca822e3-3490-427d-83ce-814fd7933a63 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:38.713285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca822e3-3490-427d-83ce-814fd7933a63 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:38.720851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e761e5f-67fc-44ad-b488-bbbe5b739ed5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:38.721489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761e5f-67fc-44ad-b488-bbbe5b739ed5 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:38.721811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761e5f-67fc-44ad-b488-bbbe5b739ed5 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:38.722288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e761e5f-67fc-44ad-b488-bbbe5b739ed5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:38.733407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e761e5f-67fc-44ad-b488-bbbe5b739ed5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:38.738134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e761e5f-67fc-44ad-b488-bbbe5b739ed5 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:38.747334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 751/1524] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:29:39.692116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15acefb-5158-4c02-bb28-d5e894cc5c40 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:39.692764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15acefb-5158-4c02-bb28-d5e894cc5c40 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:39.692764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15acefb-5158-4c02-bb28-d5e894cc5c40 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:39.694207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 774/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon May 23 13:29:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:39.699710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4471c-70e1-41cd-b81b-a9d99202a751 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:39.700224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4471c-70e1-41cd-b81b-a9d99202a751 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:39.700320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef4471c-70e1-41cd-b81b-a9d99202a751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:39.719572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3ef4471c-70e1-41cd-b81b-a9d99202a751 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:39.720848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ef4471c-70e1-41cd-b81b-a9d99202a751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:39.733465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 752/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon May 23 13:29:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:29:39.894726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15acefb-5158-4c02-bb28-d5e894cc5c40 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:39.910885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15acefb-5158-4c02-bb28-d5e894cc5c40 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:39.917046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa2a3fcb-122d-4ee9-8f44-44b5b05f9a06 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:39.917674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3fcb-122d-4ee9-8f44-44b5b05f9a06 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:39.917954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3fcb-122d-4ee9-8f44-44b5b05f9a06 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:39.918225 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa2a3fcb-122d-4ee9-8f44-44b5b05f9a06 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:39.927197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a3fcb-122d-4ee9-8f44-44b5b05f9a06 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:39.929539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa2a3fcb-122d-4ee9-8f44-44b5b05f9a06 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:39.939962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 775/1527] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:44.198242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e733e-5394-427c-a776-3a9f50dea96b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:44.198793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e733e-5394-427c-a776-3a9f50dea96b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:44.198793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6e733e-5394-427c-a776-3a9f50dea96b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:44.200264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 753/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:29:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:44.207671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb0610-5398-4560-8c56-fcc272e5deb8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:44.208226 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb0610-5398-4560-8c56-fcc272e5deb8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:44.208430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bb0610-5398-4560-8c56-fcc272e5deb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:44.227465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-68bb0610-5398-4560-8c56-fcc272e5deb8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:44.228765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-68bb0610-5398-4560-8c56-fcc272e5deb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:44.236951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 776/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:29:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:29:44.275610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e733e-5394-427c-a776-3a9f50dea96b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:44.277360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f6e733e-5394-427c-a776-3a9f50dea96b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:44.344647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6e733e-5394-427c-a776-3a9f50dea96b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:44.348384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb414dc-972e-448c-a39c-3d7324890081 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:44.349601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb414dc-972e-448c-a39c-3d7324890081 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:44.349927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb414dc-972e-448c-a39c-3d7324890081 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:44.350207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb414dc-972e-448c-a39c-3d7324890081 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:44.360764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb414dc-972e-448c-a39c-3d7324890081 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:44.362255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb414dc-972e-448c-a39c-3d7324890081 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:44.373228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 754/1530] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:46.645975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b6535-e1e9-46ef-8bdf-30ca71181941 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:46.646602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b6535-e1e9-46ef-8bdf-30ca71181941 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:46.646808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29b6535-e1e9-46ef-8bdf-30ca71181941 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:46.648240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 777/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:29:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:46.654345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940b66f-accd-4eaf-a0f6-804616b71a4e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:46.654917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940b66f-accd-4eaf-a0f6-804616b71a4e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:46.654917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6940b66f-accd-4eaf-a0f6-804616b71a4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:46.675781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6940b66f-accd-4eaf-a0f6-804616b71a4e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:46.677082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6940b66f-accd-4eaf-a0f6-804616b71a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:46.686602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 755/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:29:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:29:46.727153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b6535-e1e9-46ef-8bdf-30ca71181941 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:46.729206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d29b6535-e1e9-46ef-8bdf-30ca71181941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:46.742988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29b6535-e1e9-46ef-8bdf-30ca71181941 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:46.748032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1b93fc-0d24-4d5a-b4f7-abfb0099a71a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:46.748719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b93fc-0d24-4d5a-b4f7-abfb0099a71a service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:46.749004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b93fc-0d24-4d5a-b4f7-abfb0099a71a service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:46.749295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1b93fc-0d24-4d5a-b4f7-abfb0099a71a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:46.758791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b93fc-0d24-4d5a-b4f7-abfb0099a71a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:46.760238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1b93fc-0d24-4d5a-b4f7-abfb0099a71a service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:46.770236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 778/1533] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:50.641037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d97528-3244-458a-baac-67447ca24862 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:50.641597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d97528-3244-458a-baac-67447ca24862 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:50.641597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d97528-3244-458a-baac-67447ca24862 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:50.642989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 756/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:29:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:50.648455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b622bd-e6dd-4d12-9a04-6f16ef085db6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:50.648953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b622bd-e6dd-4d12-9a04-6f16ef085db6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:50.649061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b622bd-e6dd-4d12-9a04-6f16ef085db6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:50.667725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e9b622bd-e6dd-4d12-9a04-6f16ef085db6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:50.668981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e9b622bd-e6dd-4d12-9a04-6f16ef085db6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:50.676932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 779/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:29:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:29:50.957696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d97528-3244-458a-baac-67447ca24862 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:50.960615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15d97528-3244-458a-baac-67447ca24862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:51.001741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:29:51.019081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d97528-3244-458a-baac-67447ca24862 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:51.023357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ef8f2e-a5b6-43fa-92d0-3f247d20074a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:51.024009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef8f2e-a5b6-43fa-92d0-3f247d20074a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:51.024291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef8f2e-a5b6-43fa-92d0-3f247d20074a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:51.024580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ef8f2e-a5b6-43fa-92d0-3f247d20074a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:51.033587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef8f2e-a5b6-43fa-92d0-3f247d20074a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:51.035082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ef8f2e-a5b6-43fa-92d0-3f247d20074a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:51.045841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 757/1536] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:29:53.377072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d78d4-394e-47e4-b430-669cf5f6292d None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:53.377629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d78d4-394e-47e4-b430-669cf5f6292d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:53.377629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27d78d4-394e-47e4-b430-669cf5f6292d None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:53.379014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 780/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon May 23 13:29:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:29:53.385368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef451-da0e-4a9d-ac4b-cdfb52b7dc9a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:53.385815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef451-da0e-4a9d-ac4b-cdfb52b7dc9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:53.385921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bef451-da0e-4a9d-ac4b-cdfb52b7dc9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:53.406205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-28bef451-da0e-4a9d-ac4b-cdfb52b7dc9a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:29:53.407407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28bef451-da0e-4a9d-ac4b-cdfb52b7dc9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:53.415810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 758/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon May 23 13:29:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:29:53.721344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d78d4-394e-47e4-b430-669cf5f6292d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:29:53.723717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b27d78d4-394e-47e4-b430-669cf5f6292d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:29:53.734489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:29:53.741354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27d78d4-394e-47e4-b430-669cf5f6292d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:29:53.745044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b2a8c7-729d-4adc-866e-899992887c2d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:29:53.746281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2a8c7-729d-4adc-866e-899992887c2d service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:29:53.746281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2a8c7-729d-4adc-866e-899992887c2d service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:29:53.746281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b2a8c7-729d-4adc-866e-899992887c2d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:29:53.756963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2a8c7-729d-4adc-866e-899992887c2d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:29:53.758436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b2a8c7-729d-4adc-866e-899992887c2d service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:29:53.769566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 781/1539] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:29:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:30:03.399393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=13 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:30:03.766501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:30.626453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:30.628252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:30.629119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:30.666937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:30.668213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:30.680681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 759/1540] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:30.695688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:30.697227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:30.698034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:30.727226 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:30.728450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:30.743676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 782/1541] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:30.754044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:30.766489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f95fd-02f6-428b-81e7-0bcd90e93587 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:30.781250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:30.781250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:30.781250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:30.785064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:30.785064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:30.785064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:30.785970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 23 13:31:30.785970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:31:30.785970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:31:30.787646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:30.812350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 760/1542] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:30.821911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:30.837378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22432d4-1a29-4fc5-9a41-c221352cd992 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:30.847338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:30.848201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:30.848550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:30.848909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:30.849686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:30.851722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:31:30.851722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:31:30.851722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:31:30.853074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99241ecf-5619-477a-b240-86da2569c896 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:30.963760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 783/1543] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:30 2022] POST /identity/v3/projects => generated 381 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:30.968551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:30.980184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f1bed-e5d3-4ae0-ad37-84dc747c0e70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.056537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.057441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.057730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.058008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.058803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.059940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:31:31.059940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:31:31.059940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:31:31.061218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.092075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7362b-6355-42ea-b023-195e11b06946 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.092356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7362b-6355-42ea-b023-195e11b06946 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.092669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d7362b-6355-42ea-b023-195e11b06946 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.136099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-86d7362b-6355-42ea-b023-195e11b06946 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.137626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-86d7362b-6355-42ea-b023-195e11b06946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.150196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 784/1544] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (3 switches on core 0) May 23 13:31:31.161622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.161985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.162257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.182121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 761/1545] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:30 2022] POST /identity/v3/users => generated 291 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:31.186331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.190426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.199513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.204170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78705c68-b61c-47d6-ac4b-a98674f7c593 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.213236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 785/1546] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:31.218631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.220784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.221483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.221764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.222035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.222770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.224383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.228460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b47121-fab4-494c-839c-9b1c63e36461 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.246067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.246765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.247051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.247329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.248078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:31.248483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.248804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 762/1547] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:31.250445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.252926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.270467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 786/1548] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:31.274736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a65b05-f77c-4770-9e24-aed99f3f8bab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.280664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.285667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.286477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.286766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.287048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.287938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.289729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.298594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570bf8e2-ef92-4927-aca9-0a89d7ef5723 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.308246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d8726a-2704-4e9e-9063-99fe91b7c13b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.308888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8726a-2704-4e9e-9063-99fe91b7c13b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.309175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8726a-2704-4e9e-9063-99fe91b7c13b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.309458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d8726a-2704-4e9e-9063-99fe91b7c13b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.310172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8726a-2704-4e9e-9063-99fe91b7c13b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.311570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 763/1549] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:31.312089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d8726a-2704-4e9e-9063-99fe91b7c13b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.318143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.337890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aecb04-4a90-4725-a8b5-232ffccb8ea7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.356314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f0714c6-9a4b-4dbf-8e3c-3abb423c1128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.362500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0714c6-9a4b-4dbf-8e3c-3abb423c1128 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.362500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0714c6-9a4b-4dbf-8e3c-3abb423c1128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.362500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f0714c6-9a4b-4dbf-8e3c-3abb423c1128 admin admin] PATH_INFO: `/v3/projects/29cf54b2890e47458b0a7ff8e768c2d9/users/e936947d4bfa42ad84043506ab2bb6e8/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.379766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0714c6-9a4b-4dbf-8e3c-3abb423c1128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=29cf54b2890e47458b0a7ff8e768c2d9, user_id=e936947d4bfa42ad84043506ab2bb6e8, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.382884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f0714c6-9a4b-4dbf-8e3c-3abb423c1128 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.461034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 787/1550] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/projects => generated 389 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:31:31.462390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245748cb-1570-48a8-be03-eaad3baa28be None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.462696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245748cb-1570-48a8-be03-eaad3baa28be None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.462980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245748cb-1570-48a8-be03-eaad3baa28be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.485367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-245748cb-1570-48a8-be03-eaad3baa28be None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.486640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-245748cb-1570-48a8-be03-eaad3baa28be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.497556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 764/1551] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:31 2022] PUT /identity/v3/projects/29cf54b2890e47458b0a7ff8e768c2d9/users/e936947d4bfa42ad84043506ab2bb6e8/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:31.498874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.499186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.499489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.547102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.548375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.634848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 788/1552] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.639531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52bdf-615f-4146-8bd6-98c8c5068f75 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.640244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52bdf-615f-4146-8bd6-98c8c5068f75 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.640782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e52bdf-615f-4146-8bd6-98c8c5068f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.671788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 765/1553] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.672695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.673208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b4e52bdf-615f-4146-8bd6-98c8c5068f75 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.674664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4e52bdf-615f-4146-8bd6-98c8c5068f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.686344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bae9f9-5a0d-481a-89ba-adc3343fe89d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.689304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 789/1554] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.691004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5dfd8-33fb-42b6-946e-b4962f7645e9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.691294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5dfd8-33fb-42b6-946e-b4962f7645e9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.691674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d5dfd8-33fb-42b6-946e-b4962f7645e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.692139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540d7bb6-c347-438c-9238-77c9111ba328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.693726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d7bb6-c347-438c-9238-77c9111ba328 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.694234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d7bb6-c347-438c-9238-77c9111ba328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.694234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540d7bb6-c347-438c-9238-77c9111ba328 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.695480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d7bb6-c347-438c-9238-77c9111ba328 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.697480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540d7bb6-c347-438c-9238-77c9111ba328 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.731536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-54d5dfd8-33fb-42b6-946e-b4962f7645e9 None None] MFA Rules not processed for user `e936947d4bfa42ad84043506ab2bb6e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.732881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-54d5dfd8-33fb-42b6-946e-b4962f7645e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.809194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 790/1555] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.810609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb2f5b-3b0d-449f-a670-23d6ba9f14e6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.810907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb2f5b-3b0d-449f-a670-23d6ba9f14e6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.811195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cb2f5b-3b0d-449f-a670-23d6ba9f14e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.824496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:31.830913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-33cb2f5b-3b0d-449f-a670-23d6ba9f14e6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.832145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-33cb2f5b-3b0d-449f-a670-23d6ba9f14e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.832922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 766/1556] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/users => generated 295 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:31.834751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24d25b-7a77-4d93-ba53-280b561d5335 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.835647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24d25b-7a77-4d93-ba53-280b561d5335 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.835886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c24d25b-7a77-4d93-ba53-280b561d5335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.843861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 791/1557] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.845640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1707-d12e-4dcc-8677-183d05494329 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.846142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1707-d12e-4dcc-8677-183d05494329 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.846476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ba1707-d12e-4dcc-8677-183d05494329 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.852045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:31.858291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6c24d25b-7a77-4d93-ba53-280b561d5335 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.859579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c24d25b-7a77-4d93-ba53-280b561d5335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.867112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-69ba1707-d12e-4dcc-8677-183d05494329 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.872302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-69ba1707-d12e-4dcc-8677-183d05494329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.872455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 767/1558] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.872455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23047e78-1ccd-4218-8388-cc6d6ae2fac0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.872455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23047e78-1ccd-4218-8388-cc6d6ae2fac0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.872455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23047e78-1ccd-4218-8388-cc6d6ae2fac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.881334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 792/1559] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.881334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.881334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.881334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.893163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-23047e78-1ccd-4218-8388-cc6d6ae2fac0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.894342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-23047e78-1ccd-4218-8388-cc6d6ae2fac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.900804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.902046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.903398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 768/1560] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.904497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.904908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.905463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.914071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 793/1561] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.925704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.925704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45ab3cf9-63ca-43b7-8d61-718bf743a473 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.927195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:31.928641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:31.930528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.931233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.931521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.931808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.932511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.934512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.937938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 769/1562] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:31.938506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.950738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6b7156-3236-40c5-8171-55096e2affd2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.955971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.956693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.956977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.957249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.958021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.960247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:31.971177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 770/1563] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:31.972142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:31.982888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337d3a97-86a9-485d-859c-d04dd29270bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:31.987481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:31.988164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:31.988477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:31.988794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:31.989514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:31.989825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:31.992070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a6f716b-eba0-455e-939a-383b92d44bc9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.004633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 771/1564] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:32.006016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.006368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.006648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.024820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:32.030772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:32.035169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 772/1565] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:32.035977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.046529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2cc35-c55d-448a-93a2-2453d297dacd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.051324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.051993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.052279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.052552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.053919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:32.054258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.055914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3507da-2629-424e-898d-ed2b43e42c7b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.069973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 794/1566] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:31 2022] POST /identity/v3/projects => generated 381 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:32.076273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.080703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 773/1567] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:32.082196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a138f-e06c-4fc5-8437-ff27f298aff4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.082766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a138f-e06c-4fc5-8437-ff27f298aff4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.083056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8a138f-e06c-4fc5-8437-ff27f298aff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.107561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fc8a138f-e06c-4fc5-8437-ff27f298aff4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:32.109405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fc8a138f-e06c-4fc5-8437-ff27f298aff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:32.165292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa993db3-09bf-4571-b94a-9100e978eb23 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.172841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.173797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.174527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.174822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.175874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:32.180209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.180209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.189626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 774/1568] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:32.191406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.191851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.193050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.197908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 795/1569] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:32.199065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.212819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd200745-a3b7-4424-b724-2bfdc7efd81d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.219361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.220060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.220350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.220686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.221455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.222278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:32.223220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.223867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:32.233930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 775/1570] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:32.234227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 796/1571] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:32.235162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.235275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.247127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ec8fc-d7e7-4807-8db1-4246e51bd68e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.248274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfb6c56-6df3-4538-9cdc-2630eab7432b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.251942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.252602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.252897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.253178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.253849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.253984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:32.254203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.254524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.254814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.255098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.255806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.255932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.260021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.271119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 797/1572] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:32.273256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.291521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def7cca5-e0f6-45ce-9202-67fe162359e7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.296442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.297163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.297447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.297773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.298474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.300533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.385788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 776/1573] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:32.385788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.423557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 798/1574] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/users => generated 297 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:32.423557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.435105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61464f3-c07b-4cda-a7f4-a380014d6f5b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.464409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3fe394b-7fa6-4521-90c7-f254e3287d3c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.469184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.469857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.470144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.470426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.471126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.473256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.498426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.510560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.510560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.510560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.510560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:32.510560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.510560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.514791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 799/1575] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:32.515611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.527375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e2fa27-6750-414f-ab56-0eb1319de527 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.532199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.535709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.535709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.535709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.535709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.536698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.545584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:32.582384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:32.588997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 777/1576] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/projects => generated 377 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:32.589860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.654297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700dcdd4-d2ee-40d2-8a8f-ded32f1ce35b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.654609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 800/1577] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/projects => generated 377 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:32.655632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.718033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.718853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.719133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.719405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] PATH_INFO: `/v3/projects/611e3a80618d4d4dbfb150456f0d3502/users/b3eae736c69843ceadc6cbdb258b7195/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.725828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffd74b6-a526-4ecc-b29f-2c2f4111d492 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.731135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.731845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.732496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.732496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.733160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:32.733470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.735087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.735552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] RBAC: Authorizing `identity:create_grant(project_id=611e3a80618d4d4dbfb150456f0d3502, user_id=b3eae736c69843ceadc6cbdb258b7195, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.738283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.747591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 801/1578] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:32.749615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.760520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c09cc4b-e527-4cfb-bba3-ed688f6cd486 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.765164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.765820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.766104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.766384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.767080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.769061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.820028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 778/1579] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:32 2022] PUT /identity/v3/projects/611e3a80618d4d4dbfb150456f0d3502/users/b3eae736c69843ceadc6cbdb258b7195/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:32.821635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.857122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 802/1580] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/projects => generated 385 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:32.857122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.892437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420125c4-2feb-41ab-a0e5-4343c825a470 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.921558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648cc59a-e227-43db-84e9-8afc3c70a8b6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.926241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.926906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.927189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.927464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.929095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.930925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.933132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:32.942143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 803/1581] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:32 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:32.943186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:32.954500 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b423754b-85cb-4f4c-b258-ca627f81c4c3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:32.958485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.958485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.958485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.958817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.960112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:32.960259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.960386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:32.960651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:32.960938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:32.961634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:32.961800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:32.963545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.100672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 779/1582] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/users => generated 288 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.102211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.112712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ac01f2-0923-4857-91bb-1bb66e5ae012 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.119014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.119711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.120021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.120307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.121086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.123139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.141333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 804/1583] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:32 2022] POST /identity/v3/projects => generated 393 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.141333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.180892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:33.205977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6c47f70-dba0-45f0-b862-ebc46e64590e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.214329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df26ae00-858d-4f12-b693-591952a1b3ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.214558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 780/1584] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/users => generated 289 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.215411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ae00-858d-4f12-b693-591952a1b3ed admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.215898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ae00-858d-4f12-b693-591952a1b3ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.216372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df26ae00-858d-4f12-b693-591952a1b3ed admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.217658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26ae00-858d-4f12-b693-591952a1b3ed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.220452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.221157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df26ae00-858d-4f12-b693-591952a1b3ed admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.237037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9081e961-3d34-4aa9-bbe5-4f14bd217045 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.242965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.243632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.243915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.244192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.244933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.246836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.353065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 805/1585] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/users => generated 289 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.353265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 781/1586] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/projects => generated 371 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.353265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.353495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.353495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.353495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.373229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:33.374802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:33.421229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cda2ac8-7c23-4a1f-8a08-8550c6a2e77c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.426878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.427554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.427825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.428283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.428858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.431143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.440207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 806/1587] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:33.441370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.451489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4536608b-f04c-4a57-90ee-3b10b11eaf1f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.456585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.457491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.457776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.458054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] PATH_INFO: `/v3/projects/64c54d8c6f304cfb9357a49be01ba32a/users/d96e3bef8a9848b483ddcde7104eee93/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.474004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c54d8c6f304cfb9357a49be01ba32a, user_id=d96e3bef8a9848b483ddcde7104eee93, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.475608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:31:33.475608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:31:33.475608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:31:33.477719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.525812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 782/1588] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/users => generated 293 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.526670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.537856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fa5551a-0c89-4abc-8bb3-454602ae240b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.544032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.544864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.545352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.545754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.546725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.549088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.566208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 783/1589] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.566208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.577543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cfd0c7-f091-414a-9600-1a70fb86b742 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.583612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.585077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.585377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.585891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.586562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.588587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 807/1590] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:33 2022] PUT /identity/v3/projects/64c54d8c6f304cfb9357a49be01ba32a/users/d96e3bef8a9848b483ddcde7104eee93/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:33.593106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.593228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.656161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024439b9-3aa3-435b-b0f7-df4cd2d485e5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.660935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.661759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.662163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.662558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.663592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.665638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.672882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 784/1591] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/users => generated 297 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.673796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.679567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 808/1592] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.680359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.684335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987b0a9-81e5-4ba5-be2b-581704595585 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.690334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f7c1c2-f264-43a7-9274-ce05b5564b54 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.690480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.691183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.691513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.691741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.692530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.694729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.695788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.696465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.696787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.697074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.697771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.699697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.706746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 785/1593] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.707795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.753984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:INSERT=13 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:33.769204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31303a5b-350c-4db2-bdb5-57d05f6288b9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.773885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.774834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.775169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.775604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.786823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.788372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.797808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 786/1594] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:33.798610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.805494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 809/1595] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:33 2022] POST /identity/v3/users => generated 286 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:33.805494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.809892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612fdf0c-7f23-4750-a52c-2f8b3e808679 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.814452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.815126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.815408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.815689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.815818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b8ca6d-b98e-4023-b4f4-95661a08d157 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.816410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.818096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.821492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.822271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.822632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.822984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.823793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.825526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.831915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 787/1596] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.832870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.841831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 810/1597] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.842951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.843909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59399caa-9f82-4b4c-a614-d74ec76ee8eb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.849684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.850580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.850868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.851223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.852045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.853830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.857334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8680c5f4-1e2b-486a-b4db-edb20150d6da admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.860349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.861135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.861426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.861711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.862441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.864039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.869454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 788/1598] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.869454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.870360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:33.877508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52eee09-a5e0-4e98-bd83-2e99ee7cdb99 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.883118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 811/1599] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.883118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.886709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.887381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.887662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.887940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.888686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.890303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.893940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e659c-f59e-4a33-9bd1-56d4b96c4d7f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.898960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.899775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.900133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.900485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.904662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.904993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 789/1600] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.904993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.905183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.917390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44eff891-0965-48d4-a9c5-d9ee28737a72 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.918885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 812/1601] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.919826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.923323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.924000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.924290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.924867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.925361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.926995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.930902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e7985-a77d-4ecc-82e5-92e6e09776dd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.936125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.936856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.944407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.944407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.944407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.944407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.944850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 790/1602] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.944850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.955438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 813/1603] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:33.956146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:33.960795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f04fd86-ad1f-4535-b1a1-7ea4680f34dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.967059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.967980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.968272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.968623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] PATH_INFO: `/v3/projects/195b606304014f5189b7c349b1c3a7cb/users/3aac0dd890184a748ce19248bc9caf4d/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.969439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1d931f-9e09-4335-b5c3-4c180895e2cb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:33.974314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:33.975385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:33.975735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:33.976044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] PATH_INFO: `/v3/projects/64c54d8c6f304cfb9357a49be01ba32a/users/d96e3bef8a9848b483ddcde7104eee93/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:33.978267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64c54d8c6f304cfb9357a49be01ba32a, user_id=d96e3bef8a9848b483ddcde7104eee93, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.981099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.986605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] RBAC: Authorizing `identity:create_grant(project_id=195b606304014f5189b7c349b1c3a7cb, user_id=3aac0dd890184a748ce19248bc9caf4d, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:33.989469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:33.993751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 814/1604] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:33 2022] PUT /identity/v3/projects/64c54d8c6f304cfb9357a49be01ba32a/users/d96e3bef8a9848b483ddcde7104eee93/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.000695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.066669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72147419-0806-4222-b769-44cb0d01d5a6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.071464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.072132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.072415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.072719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.073448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.075066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.087455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 815/1605] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:33 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:34.089986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.091815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 791/1606] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:33 2022] PUT /identity/v3/projects/195b606304014f5189b7c349b1c3a7cb/users/3aac0dd890184a748ce19248bc9caf4d/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.092771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.101604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6c5a84-59c2-48d5-a6f3-c2682a9f6eb1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.163528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e71195-279b-4e02-a40f-9340097a7a72 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.168094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.169032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.169318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.169593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] PATH_INFO: `/v3/projects/3f643c268087457e88f3ddabe8715a87/users/1d30c6c8fd1248908948c1a23ebf15b8/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.169855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.170770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.171066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.171361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] PATH_INFO: `/v3/projects/6bb9e74e9b64480a80d45a03945ea51f/users/a58edb91d7b641298e40dd356eec9eac/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.189780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f643c268087457e88f3ddabe8715a87, user_id=1d30c6c8fd1248908948c1a23ebf15b8, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.192691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.199565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bb9e74e9b64480a80d45a03945ea51f, user_id=a58edb91d7b641298e40dd356eec9eac, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.202630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.295134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 816/1607] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:34 2022] PUT /identity/v3/projects/3f643c268087457e88f3ddabe8715a87/users/1d30c6c8fd1248908948c1a23ebf15b8/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.295134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.312128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 792/1608] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:34 2022] PUT /identity/v3/projects/6bb9e74e9b64480a80d45a03945ea51f/users/a58edb91d7b641298e40dd356eec9eac/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.318917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.343043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:34.379707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e2e0b7-266b-4bed-a5bc-4f9210b3f237 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.400222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:34.406371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f917c90-fed7-4bf9-b70e-8cbd990bda8b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.411926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.413535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.413889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.414233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] PATH_INFO: `/v3/projects/7649fa9a0ae4411aabf060f99e691520/users/cf30fa512142489a8dea40ce426a5f36/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.434070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] RBAC: Authorizing `identity:create_grant(project_id=7649fa9a0ae4411aabf060f99e691520, user_id=cf30fa512142489a8dea40ce426a5f36, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.435838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.447126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9651d09-731f-464f-abe3-5c93940243c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.447883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9651d09-731f-464f-abe3-5c93940243c6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.448160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9651d09-731f-464f-abe3-5c93940243c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.448447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9651d09-731f-464f-abe3-5c93940243c6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.449195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9651d09-731f-464f-abe3-5c93940243c6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.450897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9651d09-731f-464f-abe3-5c93940243c6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.462619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 817/1609] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:34 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:34.464211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6df599-1e9b-47b8-9f57-9dafe7d35edd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.464622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6df599-1e9b-47b8-9f57-9dafe7d35edd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.464989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6df599-1e9b-47b8-9f57-9dafe7d35edd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.496448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9e6df599-1e9b-47b8-9f57-9dafe7d35edd None None] MFA Rules not processed for user `d96e3bef8a9848b483ddcde7104eee93`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:34.497734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9e6df599-1e9b-47b8-9f57-9dafe7d35edd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:34.541674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 793/1610] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:34 2022] PUT /identity/v3/projects/7649fa9a0ae4411aabf060f99e691520/users/cf30fa512142489a8dea40ce426a5f36/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.541674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.578553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 818/1611] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:34.580060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13c0d6-f320-4333-9a67-70fa860d02cc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.585894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13c0d6-f320-4333-9a67-70fa860d02cc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.585894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c13c0d6-f320-4333-9a67-70fa860d02cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.613903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2c13c0d6-f320-4333-9a67-70fa860d02cc None None] MFA Rules not processed for user `3aac0dd890184a748ce19248bc9caf4d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:34.615186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2c13c0d6-f320-4333-9a67-70fa860d02cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:34.630624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-017087f7-30ba-47cc-957e-1a4227c1fece admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.636680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.637710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.637995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.638272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] PATH_INFO: `/v3/projects/c4fb50a6ddee45278948b5fed215597b/users/af29f227080147a1a157f88842d39483/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.655335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4fb50a6ddee45278948b5fed215597b, user_id=af29f227080147a1a157f88842d39483, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.658384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.684619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:34.688475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 819/1612] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:34.689942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7382f7-fc50-4ac5-9698-21dbe5542e14 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.690243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7382f7-fc50-4ac5-9698-21dbe5542e14 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.690528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7382f7-fc50-4ac5-9698-21dbe5542e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.722298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8c7382f7-fc50-4ac5-9698-21dbe5542e14 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:34.724148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8c7382f7-fc50-4ac5-9698-21dbe5542e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:34.732746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 820/1613] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:34.734079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4604b-881a-4278-bc63-51f2fb5314b8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.734400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4604b-881a-4278-bc63-51f2fb5314b8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.734639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a4604b-881a-4278-bc63-51f2fb5314b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.749997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 794/1614] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:34 2022] PUT /identity/v3/projects/c4fb50a6ddee45278948b5fed215597b/users/af29f227080147a1a157f88842d39483/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.751752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.758893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f3a4604b-881a-4278-bc63-51f2fb5314b8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:34.758893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f3a4604b-881a-4278-bc63-51f2fb5314b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:34.831715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0373c81-e97c-4ac6-8040-0020e4a61a01 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.836706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 821/1615] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:34.836706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.836706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.836706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.844560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c93ef7-7c64-4e58-a273-4aa1e9dcc618 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.845049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93ef7-7c64-4e58-a273-4aa1e9dcc618 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.845320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93ef7-7c64-4e58-a273-4aa1e9dcc618 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.845601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c93ef7-7c64-4e58-a273-4aa1e9dcc618 admin admin] PATH_INFO: `/v3/projects/bf37e72c68cc49aa95b55aeb0608ae00/users/01bad3ed00ad44a09c330b38f5829162/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.856148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:34.857437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:34.862628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c93ef7-7c64-4e58-a273-4aa1e9dcc618 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bf37e72c68cc49aa95b55aeb0608ae00, user_id=01bad3ed00ad44a09c330b38f5829162, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:34.865405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c93ef7-7c64-4e58-a273-4aa1e9dcc618 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:34.870072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 822/1616] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:34.870072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:34.930201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfde47c3-c31a-400f-a311-d9e63e04067f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:34.934017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1697b8d-87fd-486f-9cb2-74104712ac72 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:34.934713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697b8d-87fd-486f-9cb2-74104712ac72 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.934938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697b8d-87fd-486f-9cb2-74104712ac72 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.935223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1697b8d-87fd-486f-9cb2-74104712ac72 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.945933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 795/1617] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:34 2022] PUT /identity/v3/projects/bf37e72c68cc49aa95b55aeb0608ae00/users/01bad3ed00ad44a09c330b38f5829162/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:34.947988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b237-cdf8-4917-a4d6-beb02a1e461e None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:34.947988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b237-cdf8-4917-a4d6-beb02a1e461e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:34.947988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9517b237-cdf8-4917-a4d6-beb02a1e461e None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:34.949174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 796/1618] 10.222.0.70 () {54 vars in 961 bytes} [Mon May 23 13:31:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:31:34.949959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b237-cdf8-4917-a4d6-beb02a1e461e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.008133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:35.020845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9517b237-cdf8-4917-a4d6-beb02a1e461e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.025218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.025450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697b8d-87fd-486f-9cb2-74104712ac72 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.026239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.026628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.026926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1697b8d-87fd-486f-9cb2-74104712ac72 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.027041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.041360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 823/1619] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.042513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.042803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.043255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.074267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:35.075522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:35.114981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:35.116883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.118640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.129200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 797/1620] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.130106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.140020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771ea7b1-8d09-4ebb-b1da-b09d70217419 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.144147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da261180-5cb2-4cf6-b89f-ef645ed0cf55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.144997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da261180-5cb2-4cf6-b89f-ef645ed0cf55 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.145392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da261180-5cb2-4cf6-b89f-ef645ed0cf55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.145774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da261180-5cb2-4cf6-b89f-ef645ed0cf55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.147456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 824/1621] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:35.148359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.156904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da261180-5cb2-4cf6-b89f-ef645ed0cf55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.158306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da261180-5cb2-4cf6-b89f-ef645ed0cf55 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.159015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6741cbd-b211-452f-b43a-6ea88489e159 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.163939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6805f0d-6826-4e3e-962b-5d5b0b37ad86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.164798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805f0d-6826-4e3e-962b-5d5b0b37ad86 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.168897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805f0d-6826-4e3e-962b-5d5b0b37ad86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.168897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6805f0d-6826-4e3e-962b-5d5b0b37ad86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.181665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6805f0d-6826-4e3e-962b-5d5b0b37ad86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.183323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 798/1622] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.183442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6805f0d-6826-4e3e-962b-5d5b0b37ad86 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.184931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.185263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.185642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.211090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 825/1623] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.214117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.214422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.214690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.227495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] MFA Rules not processed for user `01bad3ed00ad44a09c330b38f5829162`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:35.232193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:35.241778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:35.243541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:35.306136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 799/1624] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:35.307018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.316797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bc5de1-a63b-4617-aafb-81f1abafc2d9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.326364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.327056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.327353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.327639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.336265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 826/1625] 10.222.0.70 () {58 vars in 1058 bytes} [Mon May 23 13:31:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:35.342711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.347120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.349522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.361611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c2446f-791d-4603-aa40-de0031c678bd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.373865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.375270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.375937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 800/1626] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.376333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.377365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.377847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.379151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.384600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.406759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d605a8a-05f2-40b1-95a1-9c653211f247 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.420539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.421373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.421767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.422153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.498170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.500288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.517244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 801/1627] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.518582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.550149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 827/1628] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:35 2022] POST /identity/v3/projects => generated 371 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:35.555321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.609788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f966d73b-a3c1-49a4-8a0e-c7e18214113c service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.677841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.680115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.680513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.681003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.688426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee92224-48c4-46fe-b0fd-6550227ab941 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.693992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.695509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.695509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.695896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.696881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.698953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.718851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:35.777679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.779796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.793813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 802/1629] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:35 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:35.872115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.872727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 828/1630] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:35 2022] POST /identity/v3/users => generated 292 bytes in 322 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:35.887874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78394cb9-8769-4596-8519-1c051435e862 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.894063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.895195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.895621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.896000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.897113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.899026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:35.921944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 803/1631] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:35 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:35.924211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:35.936467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53420d6e-5d2c-442f-9449-bd82e518f2c1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:35.944749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:35.945899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:35.946291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:35.946844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] PATH_INFO: `/v3/projects/03ec022175ea41df89e16db89421e74a/users/efca4134e4454ee294ef5d123cd05a5b/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:35.971618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ec022175ea41df89e16db89421e74a, user_id=efca4134e4454ee294ef5d123cd05a5b, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:35.977180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:36.090576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:36.110064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 829/1632] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:35 2022] PUT /identity/v3/projects/03ec022175ea41df89e16db89421e74a/users/efca4134e4454ee294ef5d123cd05a5b/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:36.111373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:36.136748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f399161e-559b-4fb8-9b23-b817299f06e3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:36.233840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:36.241995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d93b89-994f-40e7-aa06-faa1f9907df2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:36.243103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d93b89-994f-40e7-aa06-faa1f9907df2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.243411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d93b89-994f-40e7-aa06-faa1f9907df2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.243679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d93b89-994f-40e7-aa06-faa1f9907df2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.244484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d93b89-994f-40e7-aa06-faa1f9907df2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:36.246381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d93b89-994f-40e7-aa06-faa1f9907df2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:36.260612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 804/1633] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:36 2022] GET /identity/v3/roles => generated 1420 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:36.262952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:36.285526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a2b35d-5364-43cd-ba6c-4aa802c90820 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:36.308060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-556287c5-9883-42eb-9b04-bf610f386999 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:36.308465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556287c5-9883-42eb-9b04-bf610f386999 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.308901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556287c5-9883-42eb-9b04-bf610f386999 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.309281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-556287c5-9883-42eb-9b04-bf610f386999 admin admin] PATH_INFO: `/v3/projects/03ec022175ea41df89e16db89421e74a/users/efca4134e4454ee294ef5d123cd05a5b/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.315244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556287c5-9883-42eb-9b04-bf610f386999 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03ec022175ea41df89e16db89421e74a, user_id=efca4134e4454ee294ef5d123cd05a5b, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:36.320810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-556287c5-9883-42eb-9b04-bf610f386999 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:36.353727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 830/1634] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:36 2022] PUT /identity/v3/projects/03ec022175ea41df89e16db89421e74a/users/efca4134e4454ee294ef5d123cd05a5b/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:36.367785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.368089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.368395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.402611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] MFA Rules not processed for user `efca4134e4454ee294ef5d123cd05a5b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:36.404026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:36.437586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a5af2-c605-4202-ae2c-2c9326eba8ee None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.438783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a5af2-c605-4202-ae2c-2c9326eba8ee None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.439213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f79a5af2-c605-4202-ae2c-2c9326eba8ee None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.441167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 831/1635] 10.222.0.70 () {54 vars in 961 bytes} [Mon May 23 13:31:36 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:31:36.449258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8343760-7554-4d18-8aa5-9261a8b0108d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.449712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8343760-7554-4d18-8aa5-9261a8b0108d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.450057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8343760-7554-4d18-8aa5-9261a8b0108d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.470574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b8343760-7554-4d18-8aa5-9261a8b0108d None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:36.479341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b8343760-7554-4d18-8aa5-9261a8b0108d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:36.494002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 805/1636] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:36.507666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:36.553928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 832/1637] 10.222.0.70 () {58 vars in 1058 bytes} [Mon May 23 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:36.559942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d18685-1ac6-4ff1-a1ce-0fbab65a5a2a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.560265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d18685-1ac6-4ff1-a1ce-0fbab65a5a2a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.576909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d18685-1ac6-4ff1-a1ce-0fbab65a5a2a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.576909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 833/1638] 10.222.0.70 () {54 vars in 912 bytes} [Mon May 23 13:31:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:31:36.576909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb83d8-f1be-4b55-a0f4-07abd5ee15e7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.576909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb83d8-f1be-4b55-a0f4-07abd5ee15e7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.576909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eeb83d8-f1be-4b55-a0f4-07abd5ee15e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.577348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa7f78-6d53-4fa9-9120-308211a1ba23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:36.577348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:36.577348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:36.577348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:36.577657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:36.587418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:36.588851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5eeb83d8-f1be-4b55-a0f4-07abd5ee15e7 None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:37.037641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:37.039403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5eeb83d8-f1be-4b55-a0f4-07abd5ee15e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:37.042152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 806/1639] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:36 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:37.044906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 834/1640] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:31:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:37.045664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:37.045664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3c8b91-161b-4fd6-90db-47fefebc8e30 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:37.045664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c4ad4a-ab5c-4f93-9b0c-45569d253efc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:37.045664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4ad4a-ab5c-4f93-9b0c-45569d253efc service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:37.046262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4ad4a-ab5c-4f93-9b0c-45569d253efc service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:37.046262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c4ad4a-ab5c-4f93-9b0c-45569d253efc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:37.046262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4ad4a-ab5c-4f93-9b0c-45569d253efc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:37.046262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c4ad4a-ab5c-4f93-9b0c-45569d253efc service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:37.046262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 807/1641] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:37.134468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d1f96-07db-49fa-9331-86e931cbd2d4 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:37.134754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d1f96-07db-49fa-9331-86e931cbd2d4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:37.135134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558d1f96-07db-49fa-9331-86e931cbd2d4 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:37.136793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 835/1642] 10.222.0.70 () {54 vars in 912 bytes} [Mon May 23 13:31:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:31:37.147944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:37.148983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:37.149355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:37.170585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:37.171857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:37.182830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 808/1643] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:31:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:37.242092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d1f96-07db-49fa-9331-86e931cbd2d4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:37.260451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558d1f96-07db-49fa-9331-86e931cbd2d4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:37.269544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846285a-2e93-42f8-9873-169362ffffcd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:37.271052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846285a-2e93-42f8-9873-169362ffffcd service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:37.271419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846285a-2e93-42f8-9873-169362ffffcd service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:37.272605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846285a-2e93-42f8-9873-169362ffffcd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:37.285240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846285a-2e93-42f8-9873-169362ffffcd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:37.292222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e846285a-2e93-42f8-9873-169362ffffcd service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:37.326650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 836/1644] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:37.637691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:37.760100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39c09ef-0e18-4064-a8de-755b32f78dbf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:37.769433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:37.770493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:37.771336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:37.771336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] PATH_INFO: `/v3/users/3aac0dd890184a748ce19248bc9caf4d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:37.773738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aac0dd890184a748ce19248bc9caf4d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:37.780765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:37.780765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:38.165734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aac0dd890184a748ce19248bc9caf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:38.173480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aac0dd890184a748ce19248bc9caf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:38.189090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aac0dd890184a748ce19248bc9caf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:38.189860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aac0dd890184a748ce19248bc9caf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:38.261734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aac0dd890184a748ce19248bc9caf4d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:38.284004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 809/1645] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:31:37 2022] DELETE /identity/v3/users/3aac0dd890184a748ce19248bc9caf4d => generated 0 bytes in 647 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:38.289314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e8a6a-20fc-4e5f-963f-54e983af5681 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:38.290213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e8a6a-20fc-4e5f-963f-54e983af5681 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:38.290661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a7e8a6a-20fc-4e5f-963f-54e983af5681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:38.320487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9a7e8a6a-20fc-4e5f-963f-54e983af5681 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:38.322233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9a7e8a6a-20fc-4e5f-963f-54e983af5681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:38.405474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 837/1646] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:38.414748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:38.416331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:38.509456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28843642-4697-4a8b-9ae9-e0819db124a4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:38.514485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:38.515345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:38.515728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:38.516112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:38.539423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:38.541365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:38.574150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 810/1647] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:38.785049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcc1ae-7e9c-4d26-ad75-f5b757c76343 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:38.785210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcc1ae-7e9c-4d26-ad75-f5b757c76343 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:38.785637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25dcc1ae-7e9c-4d26-ad75-f5b757c76343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:38.824299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-25dcc1ae-7e9c-4d26-ad75-f5b757c76343 None None] MFA Rules not processed for user `1d30c6c8fd1248908948c1a23ebf15b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:38.825793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-25dcc1ae-7e9c-4d26-ad75-f5b757c76343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:38.954903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:38.968457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:38.970489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:38.995451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 838/1648] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:38 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:39.004552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491a53a0-55c3-4acb-9f48-d37673144781 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:39.008107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:39.008394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:39.008718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:39.030806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:39.036618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:39.039410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:39.039995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] PATH_INFO: `/v3/users/01bad3ed00ad44a09c330b38f5829162` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:39.042610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01bad3ed00ad44a09c330b38f5829162)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:39.044911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:39.045490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] MFA Rules not processed for user `1d30c6c8fd1248908948c1a23ebf15b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:39.046969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:39.062419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 839/1649] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:39.078015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:39.080093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:39.289238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbfd394-4791-45be-b015-20df03915e6c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:39.299287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:39.300492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:39.300492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:39.300492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:39.310830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:39.313272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:39.327386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 840/1650] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:39.420059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01bad3ed00ad44a09c330b38f5829162'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.448591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01bad3ed00ad44a09c330b38f5829162'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.488695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01bad3ed00ad44a09c330b38f5829162'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.490767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01bad3ed00ad44a09c330b38f5829162'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.507432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:39.509793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:39.522141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:39.532600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15cdc941-e9e4-44a5-ae40-90cbf61c3ca5 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:39.540336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:39.542210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:39.542509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:39.542793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] PATH_INFO: `/v3/projects/195b606304014f5189b7c349b1c3a7cb` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:39.545282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=195b606304014f5189b7c349b1c3a7cb)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:39.547281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:39.638231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-401a7cdd-7d5a-4bcf-8319-03bdd63fce51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01bad3ed00ad44a09c330b38f5829162'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.649497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 811/1651] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:31:38 2022] DELETE /identity/v3/users/01bad3ed00ad44a09c330b38f5829162 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:39.653861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:39.654179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:39.654424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:39.699116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:39.701639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:39.738516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '195b606304014f5189b7c349b1c3a7cb', 'request_id': 'req-39c97999-9756-421d-be31-67b7077c19c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.751488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '195b606304014f5189b7c349b1c3a7cb', 'request_id': 'req-39c97999-9756-421d-be31-67b7077c19c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.759370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '195b606304014f5189b7c349b1c3a7cb', 'request_id': 'req-39c97999-9756-421d-be31-67b7077c19c8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.829718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 812/1652] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:39.850178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:39.853547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] The token cache is being invalidate because project 195b606304014f5189b7c349b1c3a7cb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:31:39.854134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.855568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-39c97999-9756-421d-be31-67b7077c19c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:39.858005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 841/1653] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:31:39 2022] DELETE /identity/v3/projects/195b606304014f5189b7c349b1c3a7cb => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:31:39.859566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:39.863131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eeb92-0e21-4ab6-a196-c76c0b50454d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:39.863431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eeb92-0e21-4ab6-a196-c76c0b50454d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:39.863717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b14eeb92-0e21-4ab6-a196-c76c0b50454d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:39.962802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b14eeb92-0e21-4ab6-a196-c76c0b50454d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:39.965631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b14eeb92-0e21-4ab6-a196-c76c0b50454d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:40.067290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b2b30-3df8-4311-bf60-ffed26b229fd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.147624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 842/1654] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 285 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:40.156193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.156494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.156805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.184741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.185722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.186232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.186723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.190821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:40.192343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:40.193639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:40.215999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.217543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.224788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 843/1655] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:40.225924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.244610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 813/1656] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:40.248608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fbebb6-ee95-4fcb-9d3d-5200f8d4961a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.262235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.262943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.263236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.263516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.264268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:40.264587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.266234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.292741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 844/1657] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:31:40.294063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.307644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3538e9-3030-4d23-bf6a-30d55286a65a service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.321893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81decc9a-8780-44b1-925f-34e582460ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.322581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81decc9a-8780-44b1-925f-34e582460ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.322875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81decc9a-8780-44b1-925f-34e582460ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.323159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81decc9a-8780-44b1-925f-34e582460ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.323864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81decc9a-8780-44b1-925f-34e582460ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.326858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81decc9a-8780-44b1-925f-34e582460ca6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.471044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 814/1658] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:40 2022] POST /identity/v3/projects => generated 405 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:40.475745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.494066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c6455b-c53d-4722-a0b1-6b639c56708c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.570052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.570902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.571287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.571903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.572531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.575274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.601574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c779a91-7694-4461-be79-6f4aa453ad3f None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.601840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c779a91-7694-4461-be79-6f4aa453ad3f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.602134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c779a91-7694-4461-be79-6f4aa453ad3f None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.604290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 815/1659] 10.222.0.70 () {54 vars in 912 bytes} [Mon May 23 13:31:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:31:40.610160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.610429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.612590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.665498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:40.667171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:40.698408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 845/1660] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:40 2022] POST /identity/v3/users => generated 303 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:40.702637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.724473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3e57f8-2c68-4c64-9031-884a19673dbc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.736422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.737487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.738087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.738582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.742119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 816/1661] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:31:40 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:40.750735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.753298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.766985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 846/1662] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:31:40.771209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.785646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4505d8c-fd26-472f-83bf-6265baca70f3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.792276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.792875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.793228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.794274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.795186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.796936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.810280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 817/1663] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:40.814392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.837202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f699fc38-9882-4445-9a51-de7d4b5f9b76 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.847470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.848959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.853658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.853658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] PATH_INFO: `/v3/projects/cee536ba991a4308b245442a8840fb98/users/405b90c4a9f740ddbd88ca34c9008b7c/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.854250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:40.854250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:40.875468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:40.887059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a094043-591f-43aa-9a29-4148a96d10a1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:40.891132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cee536ba991a4308b245442a8840fb98, user_id=405b90c4a9f740ddbd88ca34c9008b7c, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.895340 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.899163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c68bd-f62f-49c5-8070-10913e231de8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:40.899855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c68bd-f62f-49c5-8070-10913e231de8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:40.900148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c68bd-f62f-49c5-8070-10913e231de8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:40.900432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c68bd-f62f-49c5-8070-10913e231de8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:40.923540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c68bd-f62f-49c5-8070-10913e231de8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:40.926659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c68bd-f62f-49c5-8070-10913e231de8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:40.941689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 818/1664] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:41.001206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 847/1665] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:40 2022] PUT /identity/v3/projects/cee536ba991a4308b245442a8840fb98/users/405b90c4a9f740ddbd88ca34c9008b7c/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:41.008099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:41.008473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:41.008847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:41.032875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:41.034381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:41.137090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 819/1666] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:41.173186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:41.175138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:41.297274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f9c8583-a72e-47d9-9514-8217189b1dd7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:41.305002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:41.306132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:41.306657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:41.307136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:41.322059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:41.325850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:41.343233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 848/1667] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:31:41.729079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:41.820545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:41.821057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:41.850622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9012982c-4835-465b-9463-b6b51a9df0a3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:41.863900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:41.864827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:41.865119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:41.865404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] PATH_INFO: `/v3/projects/bf37e72c68cc49aa95b55aeb0608ae00` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:41.870618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=bf37e72c68cc49aa95b55aeb0608ae00)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:41.872278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: WARNING py.warnings [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:31:41.872278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: warnings.warn(msg) May 23 13:31:41.872278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: May 23 13:31:41.888631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:42.366031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bf37e72c68cc49aa95b55aeb0608ae00', 'request_id': 'req-a53069fe-c396-460b-88e0-e2875dc0f7ee'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:42.382870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bf37e72c68cc49aa95b55aeb0608ae00', 'request_id': 'req-a53069fe-c396-460b-88e0-e2875dc0f7ee'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:42.415810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bf37e72c68cc49aa95b55aeb0608ae00', 'request_id': 'req-a53069fe-c396-460b-88e0-e2875dc0f7ee'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:42.577442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] The token cache is being invalidate because project bf37e72c68cc49aa95b55aeb0608ae00 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:31:42.577736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:42.580924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a53069fe-c396-460b-88e0-e2875dc0f7ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:42.583077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 820/1668] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:31:41 2022] DELETE /identity/v3/projects/bf37e72c68cc49aa95b55aeb0608ae00 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:42.587270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:42.610139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc124357-e1eb-41b1-8865-500aa149bba7 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:42.687727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:42.707058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382005b-723d-49ee-ac2f-e0f259a3cb45 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:42.707688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382005b-723d-49ee-ac2f-e0f259a3cb45 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:42.708162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7382005b-723d-49ee-ac2f-e0f259a3cb45 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:42.709910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 821/1669] 10.222.0.70 () {54 vars in 960 bytes} [Mon May 23 13:31:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 23 13:31:42.716934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:42.717326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:42.717729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:42.773111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:42.776368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:42.797515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:42.804209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:42.805352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:42.805830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] PATH_INFO: `/v3/users/efca4134e4454ee294ef5d123cd05a5b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:42.811084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efca4134e4454ee294ef5d123cd05a5b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:42.814315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: WARNING py.warnings [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 23 13:31:42.814315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: warnings.warn(msg) May 23 13:31:42.814315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: May 23 13:31:42.817884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:43.066616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 822/1670] 10.222.0.70 () {58 vars in 1057 bytes} [Mon May 23 13:31:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 351 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:43.094592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:43.155597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d3de7b-f52c-4b71-a30d-7468b7e5ba87 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:43.167688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:43.169532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:43.189216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:43.192682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:43.201351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:43.247309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efca4134e4454ee294ef5d123cd05a5b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:43.253060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efca4134e4454ee294ef5d123cd05a5b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:43.255709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efca4134e4454ee294ef5d123cd05a5b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:43.257094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:43.259493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:43.279432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 823/1671] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:43.322416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efca4134e4454ee294ef5d123cd05a5b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:43.358820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:43.362192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ae9e976e-01ac-420e-8f4a-5376b0b00a51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efca4134e4454ee294ef5d123cd05a5b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:43.865358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 849/1672] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:31:42 2022] DELETE /identity/v3/users/efca4134e4454ee294ef5d123cd05a5b => generated 0 bytes in 1273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:45.128641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:45.130391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:45.157438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfc5f8c-f551-4b92-9196-adb3747ae21e service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:45.310570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:45.311351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:45.312437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:45.312900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] PATH_INFO: `/v3/projects/03ec022175ea41df89e16db89421e74a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:45.315090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03ec022175ea41df89e16db89421e74a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:45.317093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:45.552315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:45.553656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:45.553656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:45.631943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] MFA Rules not processed for user `405b90c4a9f740ddbd88ca34c9008b7c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:45.633411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:45.686086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03ec022175ea41df89e16db89421e74a', 'request_id': 'req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:45.723894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03ec022175ea41df89e16db89421e74a', 'request_id': 'req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:45.755156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03ec022175ea41df89e16db89421e74a', 'request_id': 'req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:45.778687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 850/1673] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:45.788552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:45.790748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:45.859002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] The token cache is being invalidate because project 03ec022175ea41df89e16db89421e74a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:31:45.860933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:45.862105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3cb04239-1fdb-4226-99e4-ecabe6ddaaf3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:45.867467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 824/1674] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:31:45 2022] DELETE /identity/v3/projects/03ec022175ea41df89e16db89421e74a => generated 0 bytes in 739 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:45.873626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201d00-4bea-44e1-8f7c-35f55b4adb47 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:45.883752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201d00-4bea-44e1-8f7c-35f55b4adb47 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:45.883752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5201d00-4bea-44e1-8f7c-35f55b4adb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:45.918937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f5201d00-4bea-44e1-8f7c-35f55b4adb47 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:45.920195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5201d00-4bea-44e1-8f7c-35f55b4adb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:45.971177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:46.024010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca04140c-e8c6-45ea-bc1a-80a017e52df0 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.027815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:46.034126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.034835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.036840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.036840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.036840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.039310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.049697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 825/1675] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:46.057462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.057799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.058029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.089356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:46.090694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:46.103598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 826/1676] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:46.105189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.118807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7870fc-ef5b-4516-828d-872f04bea579 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.125757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.126908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.127524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.127974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.129425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:31:46.130497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.132950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.140660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 851/1677] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:45 2022] POST /identity/v3/projects => generated 403 bytes in 353 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:46.146440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.155632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 827/1678] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:31:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:31:46.159912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.160722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:46.190565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b5f5aa-6785-42fa-9d8b-7482d18b55cb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.308500 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d935ad1-978f-4c19-9e28-90a1a69748ef admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.311825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.312889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.314009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.314299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.316276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.321478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.323689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.324471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.324788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.327573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.328262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.333246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.454619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 852/1679] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:46 2022] POST /identity/v3/users => generated 308 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:46.458844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.486955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2acd1f7-cc2e-4d4d-aa99-c2303bcc6e19 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.495446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.496542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.497276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.497754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.498718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.501150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.510463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 828/1680] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:46 2022] POST /identity/v3/projects => generated 381 bytes in 351 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:46.515459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.535526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 853/1681] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:46.538405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.561565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0d9dff-744d-41e1-89b4-7f79947b60c3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.697811 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.698735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.699020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.701428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] PATH_INFO: `/v3/projects/b37b88efce884a62b2623ab9b28818b9/users/7fb1893a41404a30b57851025a978fd9/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.708345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d988da5-8c0d-40f6-8808-6ac66a0941d6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:46.722800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:46.723498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:46.723782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:46.724055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:46.725179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.727152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.742469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37b88efce884a62b2623ab9b28818b9, user_id=7fb1893a41404a30b57851025a978fd9, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:46.745836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:46.869276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 854/1682] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:46 2022] PUT /identity/v3/projects/b37b88efce884a62b2623ab9b28818b9/users/7fb1893a41404a30b57851025a978fd9/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 332 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:46.873320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.957874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 829/1683] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:46 2022] POST /identity/v3/users => generated 291 bytes in 442 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:46.961664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:46.976603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ea6330-f62d-472a-ae20-f7ef38a45260 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:47.046181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377f19da-4305-4384-a50f-e36d8c5833a6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:47.084494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:47.085630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:47.088178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:47.088473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:47.099624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:47.099624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:47.122266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:47.123673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:47.123797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:47.124031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:47.129156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:47.129156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:47.141060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 855/1684] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:47.148308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:47.188589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 830/1685] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:46 2022] GET /identity/v3/roles => generated 1420 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:47.189634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:47.191503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6627483-824b-4ec3-95fb-14b3481fba5e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:47.211281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:47.212510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:47.212849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:47.213134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] PATH_INFO: `/v3/projects/b37b88efce884a62b2623ab9b28818b9/users/7fb1893a41404a30b57851025a978fd9/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:47.217590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b37b88efce884a62b2623ab9b28818b9, user_id=7fb1893a41404a30b57851025a978fd9, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:47.224089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:47.226613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ac9f2-de07-4f3c-ac7f-06f15c233984 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:47.238352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 856/1686] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:47 2022] PUT /identity/v3/projects/b37b88efce884a62b2623ab9b28818b9/users/7fb1893a41404a30b57851025a978fd9/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:47.326084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b351632a-3a7c-479d-a45e-ffd696d7129d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:47.326675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351632a-3a7c-479d-a45e-ffd696d7129d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:47.327007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351632a-3a7c-479d-a45e-ffd696d7129d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:47.327287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b351632a-3a7c-479d-a45e-ffd696d7129d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:47.328122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351632a-3a7c-479d-a45e-ffd696d7129d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:47.329842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b351632a-3a7c-479d-a45e-ffd696d7129d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:47.344101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 831/1687] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:47 2022] GET /identity/v3/roles => generated 1420 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:47.349592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:47.365097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df544704-bd0e-4144-8bf0-fff8d8b58fb9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:47.384065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:47.386327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:47.387500 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:47.388730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] PATH_INFO: `/v3/projects/171941b890b34a93b6ea33ba34f744b1/users/93c828c1e31c4d56a18ee10b8f261b3d/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:47.427495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=171941b890b34a93b6ea33ba34f744b1, user_id=93c828c1e31c4d56a18ee10b8f261b3d, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:47.436292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:47.693173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 857/1688] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:47 2022] PUT /identity/v3/projects/171941b890b34a93b6ea33ba34f744b1/users/93c828c1e31c4d56a18ee10b8f261b3d/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:47.705266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dceea-9607-4633-bea8-8dd94128476c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:47.705474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dceea-9607-4633-bea8-8dd94128476c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:47.705783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316dceea-9607-4633-bea8-8dd94128476c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:47.773477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-316dceea-9607-4633-bea8-8dd94128476c None None] MFA Rules not processed for user `93c828c1e31c4d56a18ee10b8f261b3d`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:47.776689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-316dceea-9607-4633-bea8-8dd94128476c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:47.901905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 832/1689] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:47.906528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:48.026845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397192f7-a4ab-4697-b1f8-e8fd79f499f0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:48.036621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:48.038372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:48.038921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:48.039595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:48.040728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:48.043012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:48.205484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dceea-9607-4633-bea8-8dd94128476c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:48.206334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 858/1690] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:31:47 2022] POST /identity/v3/projects => generated 383 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:48.222921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316dceea-9607-4633-bea8-8dd94128476c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:48.409406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:48.410223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:48.410597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:48.410862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:48.411564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:48.413682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:48.650641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:48.714265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 833/1691] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:31:48 2022] POST /identity/v3/users => generated 298 bytes in 509 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:31:48.718553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:48.735886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1defce2-28f8-42d0-b7dd-00753b9546a1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:48.754178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:48.754991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:48.755286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:48.755569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:48.756871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:48.758544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:48.790477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:48.790934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 859/1692] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:48.808048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-597926d7-6685-4ff9-b73d-5c99e1237d04 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:48.814815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:48.815913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:48.816195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:48.816535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] PATH_INFO: `/v3/projects/93004a9f2c804a2ba3e838bdc148fd5d/users/c74fda130c6c4756b2856647af063ea0/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:48.856886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] RBAC: Authorizing `identity:create_grant(project_id=93004a9f2c804a2ba3e838bdc148fd5d, user_id=c74fda130c6c4756b2856647af063ea0, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:48.867289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:48.930952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:48.979625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 834/1693] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:48 2022] PUT /identity/v3/projects/93004a9f2c804a2ba3e838bdc148fd5d/users/c74fda130c6c4756b2856647af063ea0/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:48.986081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:49.010158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d41d1e-c4e9-4f35-80cb-640f834ffc9b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:49.207284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a307b9-12c2-4070-8421-a7f8b9ecaccc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:49.208242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a307b9-12c2-4070-8421-a7f8b9ecaccc admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:49.208737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a307b9-12c2-4070-8421-a7f8b9ecaccc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:49.209166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a307b9-12c2-4070-8421-a7f8b9ecaccc admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:49.212406 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a307b9-12c2-4070-8421-a7f8b9ecaccc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:49.217058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a307b9-12c2-4070-8421-a7f8b9ecaccc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:49.256929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:49.260031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 860/1694] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:31:48 2022] GET /identity/v3/roles => generated 1420 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:31:49.270144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff632dc-87e8-4dba-bbeb-7bc26882168d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:49.289445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:49.289445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:49.289445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:49.289445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] PATH_INFO: `/v3/projects/93004a9f2c804a2ba3e838bdc148fd5d/users/c74fda130c6c4756b2856647af063ea0/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:49.292708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93004a9f2c804a2ba3e838bdc148fd5d, user_id=c74fda130c6c4756b2856647af063ea0, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:49.298670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:49.345152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 835/1695] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:31:49 2022] PUT /identity/v3/projects/93004a9f2c804a2ba3e838bdc148fd5d/users/c74fda130c6c4756b2856647af063ea0/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:31:49.352031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:49.352328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:49.352813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:49.416932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] MFA Rules not processed for user `c74fda130c6c4756b2856647af063ea0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:49.418327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:49.598527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 861/1696] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:49.612201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:49.614111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:49.751431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7e3a2a-85f4-4930-8f5d-f1557d48a252 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:49.763115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c46c8f6-4273-4f42-8450-f6786c2924f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:49.763907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46c8f6-4273-4f42-8450-f6786c2924f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:49.764203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46c8f6-4273-4f42-8450-f6786c2924f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:49.764498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c46c8f6-4273-4f42-8450-f6786c2924f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:49.777586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46c8f6-4273-4f42-8450-f6786c2924f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:49.779673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c46c8f6-4273-4f42-8450-f6786c2924f3 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:49.796010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 836/1697] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:49 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:49.830955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:49.980846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:49.981644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:49.994776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bafb9979-ad3a-457b-b5c5-c92ad0b6948d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:50.004677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e7f39c-09b0-410b-a926-2b9c684ab7ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:50.006883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7f39c-09b0-410b-a926-2b9c684ab7ee service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:50.007693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7f39c-09b0-410b-a926-2b9c684ab7ee service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:50.010916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e7f39c-09b0-410b-a926-2b9c684ab7ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:50.150187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7f39c-09b0-410b-a926-2b9c684ab7ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:50.151783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e7f39c-09b0-410b-a926-2b9c684ab7ee service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:50.197194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 862/1698] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:49 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 217 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:51.412505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:51.424218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:51.424218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:51.479854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] MFA Rules not processed for user `b3eae736c69843ceadc6cbdb258b7195`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:51.481445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:51.496802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b3bb4-5e63-4cce-8749-cfb7ed3cea1c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:51.497274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b3bb4-5e63-4cce-8749-cfb7ed3cea1c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:51.497366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746b3bb4-5e63-4cce-8749-cfb7ed3cea1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:51.558008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-746b3bb4-5e63-4cce-8749-cfb7ed3cea1c None None] MFA Rules not processed for user `7fb1893a41404a30b57851025a978fd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:51.560327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-746b3bb4-5e63-4cce-8749-cfb7ed3cea1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:51.721588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 837/1699] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:51.764112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:51.765878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:51.936393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 863/1700] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 440 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:51.946417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:51.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc990d1-bbcc-423c-909d-973c933ea939 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:51.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc990d1-bbcc-423c-909d-973c933ea939 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:51.985941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc990d1-bbcc-423c-909d-973c933ea939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:51.988188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:52.047630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6bc990d1-bbcc-423c-909d-973c933ea939 None None] MFA Rules not processed for user `cf30fa512142489a8dea40ce426a5f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:52.052674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6bc990d1-bbcc-423c-909d-973c933ea939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:52.123211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff1eb5f-f12b-4643-97a8-c665a7711c36 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:52.160856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:52.165913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:52.166215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:52.166504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:52.241982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:52.242266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:52.297164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 838/1701] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:51 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 533 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:31:52.323268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 864/1702] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:51 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:52.590182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:52.608736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5ad782-a56a-4223-8d44-944176177d0a service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:52.623678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:52.624695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:52.625160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:52.625697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:52.652529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:52.654517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:52.676094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 839/1703] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:52.994706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62d74a-d674-42a6-826a-df742337c306 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:52.995140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62d74a-d674-42a6-826a-df742337c306 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:52.995249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b62d74a-d674-42a6-826a-df742337c306 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:53.062132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4b62d74a-d674-42a6-826a-df742337c306 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:53.063392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4b62d74a-d674-42a6-826a-df742337c306 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:53.269125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 865/1704] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:31:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:53.809966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:53.812035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:54.010887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e857adc-c449-4c4e-887c-f2e8c68b7bf2 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:54.032192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:54.033080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:54.033510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:54.033926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:54.055848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:54.057674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:54.078558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 840/1705] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:31:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 270 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:31:55.901453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:55.901453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:55.901453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:55.963119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] MFA Rules not processed for user `a58edb91d7b641298e40dd356eec9eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:55.968388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:56.072111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:56.306966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 866/1706] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:56.328833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:56.352820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:56.400912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a676831b-3598-4036-96d5-79bb3616318e service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:56.414836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:56.424308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:56.424308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:56.424308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] PATH_INFO: `/v3/users/93c828c1e31c4d56a18ee10b8f261b3d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:56.424308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] RBAC: Authorizing `identity:delete_user(user_id=93c828c1e31c4d56a18ee10b8f261b3d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:56.447607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9830ca53-e879-4673-8d2a-c616ae2fd480 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:56.458510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:56.464324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366103fe-31b7-406e-be55-9a56bbbbcc8b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:56.465074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366103fe-31b7-406e-be55-9a56bbbbcc8b service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:56.465355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366103fe-31b7-406e-be55-9a56bbbbcc8b service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:56.466295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366103fe-31b7-406e-be55-9a56bbbbcc8b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:56.500201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366103fe-31b7-406e-be55-9a56bbbbcc8b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:56.501770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366103fe-31b7-406e-be55-9a56bbbbcc8b service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:56.542952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 867/1707] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:56.544668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40c57f-34ef-49dd-889f-d00768474cb4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:56.545146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40c57f-34ef-49dd-889f-d00768474cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:56.545583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f40c57f-34ef-49dd-889f-d00768474cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:56.636676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0f40c57f-34ef-49dd-889f-d00768474cb4 None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:56.640537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0f40c57f-34ef-49dd-889f-d00768474cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:56.668505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:56.804530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 868/1708] 10.222.0.67 () {58 vars in 1015 bytes} [Mon May 23 13:31:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:56.824112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa8e7-76bc-4d82-b9be-b0e2e4c8fb5e None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:56.826130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa8e7-76bc-4d82-b9be-b0e2e4c8fb5e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:56.827024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8fa8e7-76bc-4d82-b9be-b0e2e4c8fb5e None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:56.830134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 869/1709] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:31:56 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:31:56.855723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fa8e7-76bc-4d82-b9be-b0e2e4c8fb5e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:56.858197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca8fa8e7-76bc-4d82-b9be-b0e2e4c8fb5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:56.894849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:56.899594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93c828c1e31c4d56a18ee10b8f261b3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:56.902022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8fa8e7-76bc-4d82-b9be-b0e2e4c8fb5e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:56.930183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-619ca82e-e7cf-4055-9e76-6b9c78f97a83 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:56.931755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ca82e-e7cf-4055-9e76-6b9c78f97a83 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:56.932912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ca82e-e7cf-4055-9e76-6b9c78f97a83 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:56.933298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-619ca82e-e7cf-4055-9e76-6b9c78f97a83 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:56.935882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93c828c1e31c4d56a18ee10b8f261b3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:56.963968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ca82e-e7cf-4055-9e76-6b9c78f97a83 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:56.963968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-619ca82e-e7cf-4055-9e76-6b9c78f97a83 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:56.970938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93c828c1e31c4d56a18ee10b8f261b3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:56.971919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93c828c1e31c4d56a18ee10b8f261b3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:57.004063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 870/1710] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:31:57.061796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c59a8b13-94a6-43bc-9464-cee46bc47bdb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93c828c1e31c4d56a18ee10b8f261b3d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:57.070091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 841/1711] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:31:56 2022] DELETE /identity/v3/users/93c828c1e31c4d56a18ee10b8f261b3d => generated 0 bytes in 999 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:31:57.074982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:57.087037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:57.087037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:57.107822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:57.108311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:57.108402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:57.170117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:57.176288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:57.206210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:57.214226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:57.337665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 871/1712] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:57.479115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 842/1713] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:31:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:57.573638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:57.575502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:57.626681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:57.628133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:57.654239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f080e2b-eac8-4deb-a9e0-caade30837af None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:57.679049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:57.679992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:57.680258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:57.682613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:57.715742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:57.739887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:57.784773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 843/1714] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:57 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:57.788861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e5ea66-534e-4a42-985d-e251f6997959 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:57.798693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3741c9-fc61-4fdd-9e01-8dcce0e6cc0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:57.799387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3741c9-fc61-4fdd-9e01-8dcce0e6cc0b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:57.799671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3741c9-fc61-4fdd-9e01-8dcce0e6cc0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:57.800125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3741c9-fc61-4fdd-9e01-8dcce0e6cc0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:57.828128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3741c9-fc61-4fdd-9e01-8dcce0e6cc0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:57.829664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3741c9-fc61-4fdd-9e01-8dcce0e6cc0b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:57.859996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 872/1715] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:31:58.038884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:58.040394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:58.073117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e306ce-ad21-4899-95e3-b9e058dde633 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:58.113473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:58.113473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:58.113473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:58.113473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] PATH_INFO: `/v3/users/e936947d4bfa42ad84043506ab2bb6e8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:58.114400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] RBAC: Authorizing `identity:delete_user(user_id=e936947d4bfa42ad84043506ab2bb6e8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:58.114400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:58.569186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e936947d4bfa42ad84043506ab2bb6e8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:58.600993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e936947d4bfa42ad84043506ab2bb6e8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:58.626190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e936947d4bfa42ad84043506ab2bb6e8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:58.629032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e936947d4bfa42ad84043506ab2bb6e8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:58.721484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:58.722117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:58.722308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:58.770483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b5af98b0-0a93-43e6-88c4-0f90648a4feb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e936947d4bfa42ad84043506ab2bb6e8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:58.780767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 844/1716] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:31:58 2022] DELETE /identity/v3/users/e936947d4bfa42ad84043506ab2bb6e8 => generated 0 bytes in 741 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:31:58.785636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d67a-7ee5-4230-95bd-386024082be2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:58.786187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d67a-7ee5-4230-95bd-386024082be2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:58.787482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d366d67a-7ee5-4230-95bd-386024082be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:58.842385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d366d67a-7ee5-4230-95bd-386024082be2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:58.851041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d366d67a-7ee5-4230-95bd-386024082be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:58.855497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] MFA Rules not processed for user `af29f227080147a1a157f88842d39483`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:58.858235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:59.076740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:59.150184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 845/1717] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:59.155769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:31:59.220680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 873/1718] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 499 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:31:59.236221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:59.236221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:59.236448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:59.282635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:59.284837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:59.302189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] MFA Rules not processed for user `af29f227080147a1a157f88842d39483`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:31:59.304407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:59.330039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 846/1719] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:31:59 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:31:59.342841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:59.345021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:59.608642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbcff28d-6291-4b14-8f56-4ed6592ca084 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:59.618989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:59.621199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:59.621649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:59.622037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:59.645747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:59.648956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:59.678780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 847/1720] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:31:59.679836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:31:59.681422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:31:59.712224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a6a43e-94b5-433e-b201-fe3f74d353a5 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:59.726445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:59.734429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:59.734429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:59.734429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] PATH_INFO: `/v3/projects/171941b890b34a93b6ea33ba34f744b1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:59.734429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=171941b890b34a93b6ea33ba34f744b1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:59.736898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:59.861571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea31b0-6a5e-4438-bf4a-119f1b21645b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:31:59.867370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:31:59.874272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:31:59.874272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:31:59.874272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:31:59.882065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:31:59.886204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:31:59.928964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 874/1721] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:31:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:31:59.942733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '171941b890b34a93b6ea33ba34f744b1', 'request_id': 'req-8cea1183-29bc-4f79-9f8e-e099a3f20776'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:31:59.994587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '171941b890b34a93b6ea33ba34f744b1', 'request_id': 'req-8cea1183-29bc-4f79-9f8e-e099a3f20776'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.010512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '171941b890b34a93b6ea33ba34f744b1', 'request_id': 'req-8cea1183-29bc-4f79-9f8e-e099a3f20776'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.133904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] The token cache is being invalidate because project 171941b890b34a93b6ea33ba34f744b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:00.134386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.135313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8cea1183-29bc-4f79-9f8e-e099a3f20776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.140737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 848/1722] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:31:59 2022] DELETE /identity/v3/projects/171941b890b34a93b6ea33ba34f744b1 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:00.146051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:00.181591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1039c3-c423-4bd8-91a5-93579e2426b9 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:00.327873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b779f6-41b5-4203-99f6-f9ef13e4bd6b None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:00.328897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b779f6-41b5-4203-99f6-f9ef13e4bd6b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:00.329510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b779f6-41b5-4203-99f6-f9ef13e4bd6b None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:00.331819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 849/1723] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:32:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:32:00.379753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:00.379753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:00.379753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:00.426414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] MFA Rules not processed for user `f40f1ccccf2f4b8e9c8373f984be3ddf`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:00.431038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:00.432728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:00.433065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:00.435378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:00.435737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] PATH_INFO: `/v3/users/c74fda130c6c4756b2856647af063ea0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:00.439768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c74fda130c6c4756b2856647af063ea0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:00.446974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:00.601872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 850/1724] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:32:00 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:00.603220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:00.605187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:00.628198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c69376-8da4-49a9-ba34-3ef2ff9cd37c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:00.650829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:00.650829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:00.650829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:00.650829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] PATH_INFO: `/v3/users/405b90c4a9f740ddbd88ca34c9008b7c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:00.653773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] RBAC: Authorizing `identity:delete_user(user_id=405b90c4a9f740ddbd88ca34c9008b7c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:00.660792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:00.898575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c74fda130c6c4756b2856647af063ea0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.900390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c74fda130c6c4756b2856647af063ea0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.903072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c74fda130c6c4756b2856647af063ea0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.969549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c74fda130c6c4756b2856647af063ea0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:00.994219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c74fda130c6c4756b2856647af063ea0'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:01.160485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 875/1725] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:00 2022] DELETE /identity/v3/users/c74fda130c6c4756b2856647af063ea0 => generated 0 bytes in 1016 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:01.161735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:01.163248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:01.206736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '405b90c4a9f740ddbd88ca34c9008b7c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:01.207617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:01.226936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '405b90c4a9f740ddbd88ca34c9008b7c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:01.263132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '405b90c4a9f740ddbd88ca34c9008b7c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:01.270240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '405b90c4a9f740ddbd88ca34c9008b7c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:01.369138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70a5f595-d41b-42ca-be35-ba7ee94ccc82 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '405b90c4a9f740ddbd88ca34c9008b7c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:01.380192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 851/1726] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:00 2022] DELETE /identity/v3/users/405b90c4a9f740ddbd88ca34c9008b7c => generated 0 bytes in 777 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:01.566771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa54869-a122-4a58-ad3e-f5e232ad1105 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:01.779773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:01.779773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:01.779773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:01.779773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:01.783128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:01.820353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:01.822371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586e172e-268a-4fd0-9c6c-da9dcd609590 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:01.856987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 876/1727] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:01 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 696 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:02.159306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:02.515364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca9b8-3bdb-426f-8ef4-2366972a6fdb None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:02.516048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca9b8-3bdb-426f-8ef4-2366972a6fdb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:02.516119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcca9b8-3bdb-426f-8ef4-2366972a6fdb None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:02.517847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 852/1728] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:32:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:32:02.576631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:02.577223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:02.580042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:02.651824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:02.656804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:02.937323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 877/1729] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:32:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 363 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:03.117754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca9b8-3bdb-426f-8ef4-2366972a6fdb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:03.119852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5dcca9b8-3bdb-426f-8ef4-2366972a6fdb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:03.204786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcca9b8-3bdb-426f-8ef4-2366972a6fdb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:03.214852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:03.215835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:03.216377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:03.216879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:03.233719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:03.236534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:03.267658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 853/1730] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:04.065513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:04.077056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:04.166382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:04.241078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e35e8-1380-4048-b2d0-19119d13740f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:04.251266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:04.253817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:04.257055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:04.257615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] PATH_INFO: `/v3/projects/cee536ba991a4308b245442a8840fb98` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:04.259555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cee536ba991a4308b245442a8840fb98)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:04.262411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:04.635922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:04.639035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:04.658885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cee536ba991a4308b245442a8840fb98', 'request_id': 'req-f8524163-46aa-4f73-84d9-7933387699d9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:04.683012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cee536ba991a4308b245442a8840fb98', 'request_id': 'req-f8524163-46aa-4f73-84d9-7933387699d9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:04.703831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cee536ba991a4308b245442a8840fb98', 'request_id': 'req-f8524163-46aa-4f73-84d9-7933387699d9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:04.720548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12dd8dca-fbbd-4167-a353-519d55da0490 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:04.868583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] The token cache is being invalidate because project cee536ba991a4308b245442a8840fb98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:04.868583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:04.884000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:04.893739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 878/1731] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:04 2022] DELETE /identity/v3/projects/cee536ba991a4308b245442a8840fb98 => generated 0 bytes in 833 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:04.902115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:04.902801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:04.914764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:04.916065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:04.916511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:04.917137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] PATH_INFO: `/v3/projects/93004a9f2c804a2ba3e838bdc148fd5d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:04.918909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93004a9f2c804a2ba3e838bdc148fd5d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:04.921161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:04.943402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:05.055399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8524163-46aa-4f73-84d9-7933387699d9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:05.062341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:05.063592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:05.064080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:05.064517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] PATH_INFO: `/v3/users/7fb1893a41404a30b57851025a978fd9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:05.066087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fb1893a41404a30b57851025a978fd9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:05.071230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:05.237206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93004a9f2c804a2ba3e838bdc148fd5d', 'request_id': 'req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.282992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93004a9f2c804a2ba3e838bdc148fd5d', 'request_id': 'req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.296093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93004a9f2c804a2ba3e838bdc148fd5d', 'request_id': 'req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.411913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fb1893a41404a30b57851025a978fd9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.413557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fb1893a41404a30b57851025a978fd9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.414693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fb1893a41404a30b57851025a978fd9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.424682 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] The token cache is being invalidate because project 93004a9f2c804a2ba3e838bdc148fd5d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:05.425112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.426394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.435454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 854/1732] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:04 2022] DELETE /identity/v3/projects/93004a9f2c804a2ba3e838bdc148fd5d => generated 0 bytes in 799 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:05.437654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:05.440039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:05.446143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:05.450855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fb1893a41404a30b57851025a978fd9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.467405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-63d3662c-1a08-4d19-aa75-c727e22575df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fb1893a41404a30b57851025a978fd9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:05.661577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 879/1733] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:04 2022] DELETE /identity/v3/users/7fb1893a41404a30b57851025a978fd9 => generated 0 bytes in 764 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:05.663888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130041e-a337-4937-8a12-461c86fc3df9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:05.664189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130041e-a337-4937-8a12-461c86fc3df9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:05.664518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130041e-a337-4937-8a12-461c86fc3df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:05.666099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0cc43a-00cd-449c-91ac-c69d94eb2e53 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:05.753482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4130041e-a337-4937-8a12-461c86fc3df9 None None] MFA Rules not processed for user `af29f227080147a1a157f88842d39483`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:05.754814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4130041e-a337-4937-8a12-461c86fc3df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:05.839487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:05.845539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:05.845539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:05.845539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] PATH_INFO: `/v3/projects/29cf54b2890e47458b0a7ff8e768c2d9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:05.845539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=29cf54b2890e47458b0a7ff8e768c2d9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:05.845539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:05.949103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 880/1734] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:05.951827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb960d-d687-4b54-9d97-b5d4f01dffce None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:05.961420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb960d-d687-4b54-9d97-b5d4f01dffce None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:05.961918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6fb960d-d687-4b54-9d97-b5d4f01dffce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:06.034405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f6fb960d-d687-4b54-9d97-b5d4f01dffce None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:06.036338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f6fb960d-d687-4b54-9d97-b5d4f01dffce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:06.059576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 881/1735] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:06.067452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:06.069335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:06.080482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:06.109474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '29cf54b2890e47458b0a7ff8e768c2d9', 'request_id': 'req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:06.132754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '29cf54b2890e47458b0a7ff8e768c2d9', 'request_id': 'req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:06.151747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:06.153933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:06.168926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '29cf54b2890e47458b0a7ff8e768c2d9', 'request_id': 'req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:06.354889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] The token cache is being invalidate because project 29cf54b2890e47458b0a7ff8e768c2d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:06.355195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:06.368659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:06.378250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 855/1736] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:05 2022] DELETE /identity/v3/projects/29cf54b2890e47458b0a7ff8e768c2d9 => generated 0 bytes in 938 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:06.383312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:06.390180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:06.415940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 882/1737] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 350 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:06.421569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:06.580298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3df00-25de-4861-a145-4fe035a02966 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:06.583161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc8d9f2-b0a2-4cb4-8498-cd588c8b25c6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:06.583161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:06.583161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:06.583161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:06.583832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] PATH_INFO: `/v3/users/d96e3bef8a9848b483ddcde7104eee93` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:06.583832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d96e3bef8a9848b483ddcde7104eee93)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:06.584155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:06.584155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:06.584155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:06.584155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:06.584658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:06.584842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:06.584842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:06.584842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:06.584842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 883/1738] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:06.584842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:06.609508 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf9251b-54e1-41ba-82dd-35efbb59fb3a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:06.616507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200ae3f3-bfb5-4945-832c-a3ed7e095808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:06.617746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ae3f3-bfb5-4945-832c-a3ed7e095808 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:06.618931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ae3f3-bfb5-4945-832c-a3ed7e095808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:06.619421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200ae3f3-bfb5-4945-832c-a3ed7e095808 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:06.620374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ae3f3-bfb5-4945-832c-a3ed7e095808 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:06.622743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200ae3f3-bfb5-4945-832c-a3ed7e095808 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:06.901778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 884/1739] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:06 2022] POST /identity/v3/projects => generated 379 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:06.903133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:06.903549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:06.903885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:06.953819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] MFA Rules not processed for user `b3eae736c69843ceadc6cbdb258b7195`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:06.955537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:06.981488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd96e3bef8a9848b483ddcde7104eee93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:07.000750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd96e3bef8a9848b483ddcde7104eee93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:07.050669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd96e3bef8a9848b483ddcde7104eee93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:07.060730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd96e3bef8a9848b483ddcde7104eee93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:07.113456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 885/1740] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:06 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:07.115961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:07.238280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c799088-0a2e-4c33-9269-db12adc371b7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:07.246199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd96e3bef8a9848b483ddcde7104eee93'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:07.246628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f021c710-ebc0-420e-9fee-b84c192dc576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:07.247478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021c710-ebc0-420e-9fee-b84c192dc576 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:07.247766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021c710-ebc0-420e-9fee-b84c192dc576 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:07.248045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f021c710-ebc0-420e-9fee-b84c192dc576 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:07.249165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021c710-ebc0-420e-9fee-b84c192dc576 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:07.252952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f021c710-ebc0-420e-9fee-b84c192dc576 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:07.258179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 856/1741] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:06 2022] DELETE /identity/v3/users/d96e3bef8a9848b483ddcde7104eee93 => generated 0 bytes in 873 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:07.260230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:07.273765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:07.387150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 886/1742] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:07 2022] POST /identity/v3/users => generated 290 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:07.392244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6acae5-1237-4681-8b7b-eb6059d8d64f None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:07.392760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6acae5-1237-4681-8b7b-eb6059d8d64f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:07.393086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6acae5-1237-4681-8b7b-eb6059d8d64f None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:07.403878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 887/1743] 10.222.0.70 () {54 vars in 912 bytes} [Mon May 23 13:32:07 2022] GET /identity => generated 270 bytes in 16 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:32:07.405220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6acae5-1237-4681-8b7b-eb6059d8d64f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:07.410638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0d6acae5-1237-4681-8b7b-eb6059d8d64f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:07.470714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea919d91-9f07-42df-9e56-7982d23b8d29 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:07.487581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:07.489284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:07.496478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:07.496478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] PATH_INFO: `/v3/projects/b37b88efce884a62b2623ab9b28818b9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:07.496478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b37b88efce884a62b2623ab9b28818b9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:07.496478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:07.667551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6acae5-1237-4681-8b7b-eb6059d8d64f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:07.667551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60158ac-6029-43f0-aa53-f1f86d252a8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:07.667551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60158ac-6029-43f0-aa53-f1f86d252a8e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:07.667551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60158ac-6029-43f0-aa53-f1f86d252a8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:07.668044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60158ac-6029-43f0-aa53-f1f86d252a8e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:07.668044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60158ac-6029-43f0-aa53-f1f86d252a8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:07.695148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60158ac-6029-43f0-aa53-f1f86d252a8e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:07.724021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 888/1744] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:07.724021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:07.724021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:07.724021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:07.773937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:07.775563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:07.964992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 889/1745] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:32:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:07.966691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:08.025447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6a22a-6bb6-4113-8e41-44a277c01809 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:08.054615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:08.055762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:08.056537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:08.057053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:08.058365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:08.075017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:08.098700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 890/1746] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:07 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:08.099602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:08.117167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:08.325564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bb6bd8-080e-4d3b-a5fc-479c49904d90 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:08.325564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:08.325564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:08.325564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:08.326870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:08.326870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:08.326870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:08.326870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 891/1747] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:08.326870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:08.326870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1523b0a-d533-47d3-8b9b-74fdbcd55fb0 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:08.327316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f18d699-322c-486f-befa-2834c933c287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:08.327316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18d699-322c-486f-befa-2834c933c287 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:08.327316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18d699-322c-486f-befa-2834c933c287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:08.327316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f18d699-322c-486f-befa-2834c933c287 admin admin] PATH_INFO: `/v3/projects/e1b81510ee63469d91d223e6bd4afad8/users/74dc7d17d44f4386afe97980c66cc617/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:08.327582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18d699-322c-486f-befa-2834c933c287 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1b81510ee63469d91d223e6bd4afad8, user_id=74dc7d17d44f4386afe97980c66cc617, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:08.327582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f18d699-322c-486f-befa-2834c933c287 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:09.721817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 892/1748] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:08 2022] PUT /identity/v3/projects/e1b81510ee63469d91d223e6bd4afad8/users/74dc7d17d44f4386afe97980c66cc617/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 1498 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:09.721817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d66c3-a90f-4a1d-b8d7-706aceed5416 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:09.721817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d66c3-a90f-4a1d-b8d7-706aceed5416 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:09.721817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2d66c3-a90f-4a1d-b8d7-706aceed5416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:09.722714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b37b88efce884a62b2623ab9b28818b9', 'request_id': 'req-befa035b-4f28-4efe-a0f8-b004fe141a0b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:09.738097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b37b88efce884a62b2623ab9b28818b9', 'request_id': 'req-befa035b-4f28-4efe-a0f8-b004fe141a0b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:09.757897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fb2d66c3-a90f-4a1d-b8d7-706aceed5416 None None] MFA Rules not processed for user `cf30fa512142489a8dea40ce426a5f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:09.758976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b37b88efce884a62b2623ab9b28818b9', 'request_id': 'req-befa035b-4f28-4efe-a0f8-b004fe141a0b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:09.762119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fb2d66c3-a90f-4a1d-b8d7-706aceed5416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:09.904194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] The token cache is being invalidate because project b37b88efce884a62b2623ab9b28818b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:09.904490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:09.905246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-befa035b-4f28-4efe-a0f8-b004fe141a0b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:09.907922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 857/1749] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:07 2022] DELETE /identity/v3/projects/b37b88efce884a62b2623ab9b28818b9 => generated 0 bytes in 2652 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:09.916672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:09.916672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:09.916672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.024517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] MFA Rules not processed for user `74dc7d17d44f4386afe97980c66cc617`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:10.024517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:10.025510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 893/1750] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:10.030881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718552ef-5042-4dc2-ba44-0c29ca8ae939 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:10.031448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718552ef-5042-4dc2-ba44-0c29ca8ae939 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:10.031941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718552ef-5042-4dc2-ba44-0c29ca8ae939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.117845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:10.159105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-718552ef-5042-4dc2-ba44-0c29ca8ae939 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:10.166694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-718552ef-5042-4dc2-ba44-0c29ca8ae939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:10.243164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 858/1751] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:09 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 334 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:10.246939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:10.250672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:10.352915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 894/1752] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:10.363042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d81a94-c986-425a-9477-54c15420e63e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:10.363341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d81a94-c986-425a-9477-54c15420e63e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:10.363691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d81a94-c986-425a-9477-54c15420e63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.389148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-14d81a94-c986-425a-9477-54c15420e63e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:10.391622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-14d81a94-c986-425a-9477-54c15420e63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:10.396999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d928fa94-c0cb-4ab9-9b6f-0ca7c126e7ab None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:10.402911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 895/1753] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:10.406160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:10.406954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:10.407251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:10.407545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.408000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d81a94-c986-425a-9477-54c15420e63e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:10.408286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:10.410539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:10.449237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d81a94-c986-425a-9477-54c15420e63e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:10.457870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:10.460040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:10.460040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:10.460269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.461570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:10.462117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:10.465304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:10.569229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 896/1754] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:10.577548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:10.635513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 859/1755] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:10 2022] POST /identity/v3/projects => generated 379 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:10.635513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:10.645612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d3d165b-096e-4714-aa0e-071647552bb2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:10.797177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9d25bd-0979-407f-a05b-2021f739ccb4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:10.815093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ba24de-78e4-4222-b3eb-e364703acfe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:10.818129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba24de-78e4-4222-b3eb-e364703acfe1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:10.819475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba24de-78e4-4222-b3eb-e364703acfe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:10.821347 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ba24de-78e4-4222-b3eb-e364703acfe1 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.825865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba24de-78e4-4222-b3eb-e364703acfe1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:10.825865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ba24de-78e4-4222-b3eb-e364703acfe1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:10.952872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:10.954430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:10.954986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:10.955429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:10.957255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:10.960081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:11.012522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=2,keystone:SELECT=96,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:11.077001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 860/1756] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:10 2022] POST /identity/v3/users => generated 290 bytes in 443 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:11.077001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:11.077001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:11.077248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:11.121831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] MFA Rules not processed for user `a58edb91d7b641298e40dd356eec9eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:11.121831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:11.157951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 897/1757] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:10 2022] POST /identity/v3/projects => generated 357 bytes in 581 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:11.159200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:11.163104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:11.264247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 861/1758] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:11 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:11.264247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:11.311497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dadcc74-d11a-4b16-999b-cd4628243ccf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:11.381606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24117914-d86d-4b28-97ea-1b84c5fe9ce3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:11.405635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:11.406473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:11.406763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:11.407098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] PATH_INFO: `/v3/projects/64c54d8c6f304cfb9357a49be01ba32a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:11.409693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] RBAC: Authorizing `identity:delete_project(project_id=64c54d8c6f304cfb9357a49be01ba32a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:11.411650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:11.419373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:11.420054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:11.420532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:11.425312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:11.425312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:11.425312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:11.441631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 862/1759] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:11.443976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:11.474342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4caf198-574b-49ab-bc62-55d0eb6a1e76 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:11.488921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:11.491592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:11.494424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:11.494424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:11.494834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:11.497294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:11.637196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64c54d8c6f304cfb9357a49be01ba32a', 'request_id': 'req-fd09a7bd-0a5c-4015-874e-273a1ece941e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:11.651058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64c54d8c6f304cfb9357a49be01ba32a', 'request_id': 'req-fd09a7bd-0a5c-4015-874e-273a1ece941e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:11.658954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64c54d8c6f304cfb9357a49be01ba32a', 'request_id': 'req-fd09a7bd-0a5c-4015-874e-273a1ece941e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:11.810942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 863/1760] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:11 2022] POST /identity/v3/users => generated 279 bytes in 367 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:11.811745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:11.826351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] The token cache is being invalidate because project 64c54d8c6f304cfb9357a49be01ba32a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:11.826889 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:11.828032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:11.832933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 898/1761] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:11 2022] DELETE /identity/v3/projects/64c54d8c6f304cfb9357a49be01ba32a => generated 0 bytes in 674 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:11.834171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:11.840641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:11.854966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09791840-1007-4cb4-8213-539b19497cfb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:11.990215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc63d6f1-b0e0-4cb3-8470-c65e15c6a70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:11.991203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63d6f1-b0e0-4cb3-8470-c65e15c6a70f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:11.991613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63d6f1-b0e0-4cb3-8470-c65e15c6a70f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:11.995625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc63d6f1-b0e0-4cb3-8470-c65e15c6a70f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:11.995625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63d6f1-b0e0-4cb3-8470-c65e15c6a70f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:11.995625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc63d6f1-b0e0-4cb3-8470-c65e15c6a70f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.023424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 864/1762] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:12.029172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd09a7bd-0a5c-4015-874e-273a1ece941e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.043627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.049021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.049021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.049280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.049280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.049280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.053142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.054310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.056072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.073905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:12.076180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:12.093966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 899/1763] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:11 2022] GET /identity/v3/roles => generated 1420 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:12.094091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.098485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 865/1764] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:12.099637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.104970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:12.152924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941651f2-aac5-4cc0-9449-0285a370cbe4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.154125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e466c608-b6f2-4f54-9430-0b8746086bbf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.162121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.163729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.167981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3d2e86-3aee-4762-878b-5eaa4061cb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.168149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.168149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] PATH_INFO: `/v3/projects/8f70aced908a46528b4cf599e0344b40/users/f5ec8d99b0494911ad6904a4dab44991/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.168316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2e86-3aee-4762-878b-5eaa4061cb7e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.168316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2e86-3aee-4762-878b-5eaa4061cb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.168316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3d2e86-3aee-4762-878b-5eaa4061cb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.168316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d2e86-3aee-4762-878b-5eaa4061cb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.168316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3d2e86-3aee-4762-878b-5eaa4061cb7e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.193400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f70aced908a46528b4cf599e0344b40, user_id=f5ec8d99b0494911ad6904a4dab44991, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.200015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.208087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 866/1765] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:12 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:12.211262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.211811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.212313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.234205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:12.235448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:12.248522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 867/1766] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:12.248522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.257290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cefb03-2891-4347-a56b-10633de464c5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.282874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752ef8d-7c95-4698-9e55-d681e00896a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.284037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752ef8d-7c95-4698-9e55-d681e00896a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.284671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752ef8d-7c95-4698-9e55-d681e00896a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.285041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752ef8d-7c95-4698-9e55-d681e00896a6 admin admin] PATH_INFO: `/v3/projects/44e40c5dd9a3441189c3cbd17c7b45ae/users/f403119bf4dc4423be4fc15c4badcc6c/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.315369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752ef8d-7c95-4698-9e55-d681e00896a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44e40c5dd9a3441189c3cbd17c7b45ae, user_id=f403119bf4dc4423be4fc15c4badcc6c, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.319848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752ef8d-7c95-4698-9e55-d681e00896a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.349473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 900/1767] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:12 2022] PUT /identity/v3/projects/8f70aced908a46528b4cf599e0344b40/users/f5ec8d99b0494911ad6904a4dab44991/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:12.359876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.442067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a8be2ea-26bc-49bf-a8f3-b015812dde23 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.448331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.449368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.449834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.450319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.451252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:12.451811 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.453990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.467314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 868/1768] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:12 2022] PUT /identity/v3/projects/44e40c5dd9a3441189c3cbd17c7b45ae/users/f403119bf4dc4423be4fc15c4badcc6c/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:12.469332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e514-3573-4725-8398-9c6d286d25de None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.469851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e514-3573-4725-8398-9c6d286d25de None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.471151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 901/1769] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 121 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:12.471547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a84e514-3573-4725-8398-9c6d286d25de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.476479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.520911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7a84e514-3573-4725-8398-9c6d286d25de None None] MFA Rules not processed for user `f5ec8d99b0494911ad6904a4dab44991`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:12.523095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b40ded8-e80d-43f0-8cea-fd632304882c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.528924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7a84e514-3573-4725-8398-9c6d286d25de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:12.663226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 869/1770] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:12.666477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.677985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.677985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.679101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.679101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.679380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.679756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.681915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.683534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.710110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] MFA Rules not processed for user `f403119bf4dc4423be4fc15c4badcc6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:12.711561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:12.800745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 870/1771] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:12.801458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.816140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c14bd4-708e-4e1f-aad3-60497314dd9a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.832346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.832346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.832346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.832346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.832669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 902/1772] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/projects => generated 371 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:12.832953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:12.836206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.838681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:12.917387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234b03a7-e42f-42cb-8d02-9dda2661f8f5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:12.933776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:12.933776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:12.933776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:12.933776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:12.934090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:12.936379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.012113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 871/1773] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/projects => generated 377 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:13.018057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.094162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 903/1774] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:12 2022] POST /identity/v3/projects => generated 355 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:13.095912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.123869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e2e242-c54f-4609-9a1c-7029c0d1db22 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.247421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186f4bb8-fe3e-4831-9c9a-3f510491da4b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.260773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.261950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.262321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.262605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.263387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.263557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.264675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.265085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.265518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.266071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.266913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.270492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.378320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 904/1775] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:13 2022] POST /identity/v3/users => generated 295 bytes in 283 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:13.379605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.385033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 872/1776] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:13 2022] POST /identity/v3/users => generated 286 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:13.386211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.399691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804acccd-cfce-47c6-958c-bc5fd27ee16a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.402626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b13017-6e42-4f0e-9cea-a9e6e2e4392f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.409930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.410623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.411020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.411246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.411865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.412037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.413376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.413728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.414243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.414964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.415094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.417089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.435723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 873/1777] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:13.436558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.457279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d656c3c-5252-4f62-8170-d50d3c48e0ff admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.463200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.463954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.464243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.464520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.465394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.466985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.479983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 874/1778] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:13.481084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.494107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea61e9b6-8254-4414-8dc8-6f570ce0da55 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.499332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.500248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.500549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.500851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] PATH_INFO: `/v3/projects/feb2ee4c4de34ef0b214639fb2226ed0/users/fa20ce99edec4485bd644e1225f76548/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.527488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb2ee4c4de34ef0b214639fb2226ed0, user_id=fa20ce99edec4485bd644e1225f76548, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.532233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.564317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 905/1779] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:13 2022] POST /identity/v3/users => generated 284 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:13.566039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.582820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b77132-d588-4c9f-b499-82d58a506025 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.591105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.596505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.596505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.596505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.596505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.597287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.623600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 906/1780] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:13.624948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.640798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57035b2e-c2f7-4822-9f6b-d0ded9056075 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.651940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.653865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.653865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.653865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.654167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.655519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.668413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 907/1781] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:13.669342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.680620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254fe6c6-5257-4426-808d-c0a0566a902f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.686183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e529d6-9b62-44c7-8bec-a7987af47efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.687337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e529d6-9b62-44c7-8bec-a7987af47efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.687625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e529d6-9b62-44c7-8bec-a7987af47efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.687905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e529d6-9b62-44c7-8bec-a7987af47efe admin admin] PATH_INFO: `/v3/projects/a3f1f93169544b91b4d05f9af07a0f61/users/d877c29d385d4d5fac2b07efdc40197f/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.691965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 875/1782] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:13 2022] PUT /identity/v3/projects/feb2ee4c4de34ef0b214639fb2226ed0/users/fa20ce99edec4485bd644e1225f76548/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:13.695773 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.712781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e529d6-9b62-44c7-8bec-a7987af47efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=a3f1f93169544b91b4d05f9af07a0f61, user_id=d877c29d385d4d5fac2b07efdc40197f, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.715488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e529d6-9b62-44c7-8bec-a7987af47efe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.768409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b9bb95-eae5-438b-909c-96071b2aee25 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:13.774445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:13.775348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.775639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.775914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] PATH_INFO: `/v3/projects/eaa4ae14a9bd47eba313542b4a488ce4/users/8048bccaba5042aebe0e8bff99a9a7b5/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.794982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa4ae14a9bd47eba313542b4a488ce4, user_id=8048bccaba5042aebe0e8bff99a9a7b5, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:13.800461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:13.919398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 876/1784] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:13 2022] PUT /identity/v3/projects/eaa4ae14a9bd47eba313542b4a488ce4/users/8048bccaba5042aebe0e8bff99a9a7b5/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:13.920368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:13.926268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 908/1784] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:13 2022] PUT /identity/v3/projects/a3f1f93169544b91b4d05f9af07a0f61/users/d877c29d385d4d5fac2b07efdc40197f/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:13.928375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:13.928961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:13.929378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:13.953201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:13.954378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:14.018575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a4ac7b-9da2-400c-ada0-c0798638b268 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.026015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.028403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.028403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.028403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.028403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.031204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.035007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 909/1785] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:14.035949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.057497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 877/1786] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:13 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:14.057497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.057497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:14.063255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca3e0f16-0b9f-490d-b281-56c77e2e7386 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.071048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.071927 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.072472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.072884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.073848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.075271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.093312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 910/1787] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:14 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:14.094267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.108107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8052a90b-5c61-40c0-8f56-449e9e2c4aba admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.120397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.122588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.122588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.122588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] PATH_INFO: `/v3/projects/feb2ee4c4de34ef0b214639fb2226ed0/users/fa20ce99edec4485bd644e1225f76548/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.124483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb2ee4c4de34ef0b214639fb2226ed0, user_id=fa20ce99edec4485bd644e1225f76548, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.127369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.138291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26d2fe68-9a09-4d80-be16-763f68417dd5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.144830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 911/1788] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:14 2022] PUT /identity/v3/projects/feb2ee4c4de34ef0b214639fb2226ed0/users/fa20ce99edec4485bd644e1225f76548/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:14.144830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.230370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c654ed-2dff-4518-8913-d432e5769b30 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.230715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c654ed-2dff-4518-8913-d432e5769b30 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.231297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c654ed-2dff-4518-8913-d432e5769b30 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.231297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c654ed-2dff-4518-8913-d432e5769b30 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.253960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7192001-f130-44bb-800d-f6d161885649 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.268183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372514e4-4bb7-45ff-b387-d1a782e571f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.270537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372514e4-4bb7-45ff-b387-d1a782e571f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.277238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372514e4-4bb7-45ff-b387-d1a782e571f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.277238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372514e4-4bb7-45ff-b387-d1a782e571f5 admin admin] PATH_INFO: `/v3/projects/eaa4ae14a9bd47eba313542b4a488ce4/users/8048bccaba5042aebe0e8bff99a9a7b5/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.277610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372514e4-4bb7-45ff-b387-d1a782e571f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eaa4ae14a9bd47eba313542b4a488ce4, user_id=8048bccaba5042aebe0e8bff99a9a7b5, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.284687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-372514e4-4bb7-45ff-b387-d1a782e571f5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.321082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c654ed-2dff-4518-8913-d432e5769b30 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.322601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c654ed-2dff-4518-8913-d432e5769b30 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.325219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 912/1789] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:14 2022] PUT /identity/v3/projects/eaa4ae14a9bd47eba313542b4a488ce4/users/8048bccaba5042aebe0e8bff99a9a7b5/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:14.329286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.329613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.329845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.336166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 878/1790] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:14.337784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.338066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.338330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.379689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] MFA Rules not processed for user `8048bccaba5042aebe0e8bff99a9a7b5`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:14.381740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:14.397436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] MFA Rules not processed for user `fa20ce99edec4485bd644e1225f76548`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:14.397436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:14.485398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 879/1791] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:14.501893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.504870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:14.511902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 913/1792] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:14 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:14.526210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=18,keystone:SELECT=80,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:14.529386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.529719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:14.614439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fa1951-f9e1-46d5-9df5-88b9192cb082 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.619686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.620022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6cf298-b31d-4599-830d-8bc394be4283 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.620307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.620614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.620895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.624077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.624755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.625046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.625327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.640842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.641000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:14.642475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.643423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:14.657021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 880/1793] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:14.673934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 914/1794] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:14.903454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:14.917147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c50a3bf-ce9e-474f-8eda-eaad1f619a6f service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:14.924604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:14.926229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:14.926613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:14.926990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:14.930061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:15.046086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:15.047792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f80a8bb-c534-4dbb-a494-488b2048fca9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:15.061428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 881/1795] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:14 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:15.085567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:15.094799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f4750b-e7f9-430e-b14a-5e4c73419fd2 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:15.098949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:15.099628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:15.099940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:15.100220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:15.167393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:15.168920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:15.181552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 915/1796] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:15.314591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:17.022552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:17.023344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:17.023707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:17.090039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] MFA Rules not processed for user `d877c29d385d4d5fac2b07efdc40197f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:17.099124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:17.340359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 882/1797] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 319 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:17.345181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:17.509513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051822a4-0359-426a-9466-89a74d66ebe6 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:17.519886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:17.521006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:17.521636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:17.526648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:17.526648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:17.526648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:17.668946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 916/1798] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:17 2022] POST /identity/v3/projects => generated 373 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:17.672177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:17.689603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f64e67f-6472-49cf-94a9-71f7fc7150df None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:17.790380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:17.791140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:17.791504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:17.791818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:17.792505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:17.794621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:17.986881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 883/1799] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:17 2022] POST /identity/v3/users => generated 293 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:17.990635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:18.008690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd9e1d0d-842f-4d32-8c2c-9e2ad13a4b70 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:18.019869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:18.021140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:18.022431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:18.022959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:18.024653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:18.026402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:18.044903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 917/1800] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:17 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:18.051190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:18.069644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2be9d3-c2d4-467a-b3a9-54f709cd2202 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:18.079491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:18.081567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:18.082752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:18.083275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] PATH_INFO: `/v3/projects/9985fe88562c4e33a10a53be41fc7364/users/cba526554d8149ebb6345e988c539c04/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:18.113362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9985fe88562c4e33a10a53be41fc7364, user_id=cba526554d8149ebb6345e988c539c04, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:18.117044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:18.216728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:18.265640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 884/1801] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:18 2022] PUT /identity/v3/projects/9985fe88562c4e33a10a53be41fc7364/users/cba526554d8149ebb6345e988c539c04/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:18.270222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:18.284170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c258032f-ce03-42eb-b3d9-a263b4660c36 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:18.388077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b89c1d-1dea-498b-bd39-8dca77df9899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:18.390213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b89c1d-1dea-498b-bd39-8dca77df9899 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:18.390742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b89c1d-1dea-498b-bd39-8dca77df9899 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:18.391540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b89c1d-1dea-498b-bd39-8dca77df9899 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:18.392993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b89c1d-1dea-498b-bd39-8dca77df9899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:18.395304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b89c1d-1dea-498b-bd39-8dca77df9899 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:18.446071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 918/1802] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:18 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:18.446229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:18.460416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff75b58-ce80-4e09-abc1-b23eeaaf263b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:18.473624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39769a5-580f-428c-b415-4583cbb8c977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:18.474578 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39769a5-580f-428c-b415-4583cbb8c977 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:18.477404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39769a5-580f-428c-b415-4583cbb8c977 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:18.477404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39769a5-580f-428c-b415-4583cbb8c977 admin admin] PATH_INFO: `/v3/projects/9985fe88562c4e33a10a53be41fc7364/users/cba526554d8149ebb6345e988c539c04/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:18.485718 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39769a5-580f-428c-b415-4583cbb8c977 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9985fe88562c4e33a10a53be41fc7364, user_id=cba526554d8149ebb6345e988c539c04, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:18.488661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39769a5-580f-428c-b415-4583cbb8c977 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:18.522861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 885/1803] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:18 2022] PUT /identity/v3/projects/9985fe88562c4e33a10a53be41fc7364/users/cba526554d8149ebb6345e988c539c04/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:18.690706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:18.995718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5396d19-165b-4678-a4af-96f5b196691a None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:18.996278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5396d19-165b-4678-a4af-96f5b196691a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:18.998063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5396d19-165b-4678-a4af-96f5b196691a None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:19.000139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 919/1804] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:32:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:32:19.008633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:19.013029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:19.013029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:19.031567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:19.033203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:19.310832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 886/1805] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:32:19 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 303 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:19.604360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5396d19-165b-4678-a4af-96f5b196691a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:19.606880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d5396d19-165b-4678-a4af-96f5b196691a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:19.637153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5396d19-165b-4678-a4af-96f5b196691a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:19.643936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8c54e-8415-40f3-b150-5e195630bcd7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:19.644639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8c54e-8415-40f3-b150-5e195630bcd7 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:19.645087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8c54e-8415-40f3-b150-5e195630bcd7 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:19.645510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8c54e-8415-40f3-b150-5e195630bcd7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:19.657499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8c54e-8415-40f3-b150-5e195630bcd7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:19.659067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8c54e-8415-40f3-b150-5e195630bcd7 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:19.670777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 920/1806] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:21.049804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:22.002866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:22.139065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed9f7e-e397-4ff0-a900-912ad27b26c2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:22.163684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:22.164795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:22.165209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:22.165623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] PATH_INFO: `/v3/users/74dc7d17d44f4386afe97980c66cc617` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:22.167556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74dc7d17d44f4386afe97980c66cc617)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:22.170485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:22.392957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74dc7d17d44f4386afe97980c66cc617'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:22.400971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74dc7d17d44f4386afe97980c66cc617'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:22.433532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74dc7d17d44f4386afe97980c66cc617'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:22.438098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74dc7d17d44f4386afe97980c66cc617'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:22.586568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46623784-7d92-49fd-819b-1678e0dc8af5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74dc7d17d44f4386afe97980c66cc617'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:22.595829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 887/1807] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:22 2022] DELETE /identity/v3/users/74dc7d17d44f4386afe97980c66cc617 => generated 0 bytes in 592 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:22.599149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:22.599933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:22.599933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:22.639855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:22.641580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:22.708703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9f0b-2892-43fe-a086-b01434729d07 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:22.709506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9f0b-2892-43fe-a086-b01434729d07 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:22.709808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-080d9f0b-2892-43fe-a086-b01434729d07 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:22.715558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 888/1808] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:32:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:32:22.760503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9cfa3-0398-49e1-8327-57af479a35b8 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:22.760839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9cfa3-0398-49e1-8327-57af479a35b8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:22.761101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff9cfa3-0398-49e1-8327-57af479a35b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:22.810592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:22.814636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 921/1809] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:22.826814 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-eff9cfa3-0398-49e1-8327-57af479a35b8 None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:22.828069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eff9cfa3-0398-49e1-8327-57af479a35b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:23.026739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:23.033861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:23.137541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 889/1810] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:32:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 378 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:23.235621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9449ac-2998-456d-b486-09ee0a1474f2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:23.241684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:23.242679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:23.243388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:23.244000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:23.255137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:23.257178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:23.269928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 922/1811] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:23.335049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:23.335457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:23.335711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:23.399102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] MFA Rules not processed for user `cba526554d8149ebb6345e988c539c04`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:23.403430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:23.566298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 890/1812] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:23 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:23.575124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:23.576771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:23.624636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:23.626078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:23.645350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c488506-614b-46ea-9a1a-8d1b59c9bc7e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:23.652439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:23.653216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:23.653515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:23.653797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:23.668336 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:23.670017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:23.671032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90b15ce2-df79-4a50-be5c-798d0d22a05f service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:23.677155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:23.678458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:23.678832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:23.679174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:23.681320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:23.683202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 891/1813] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:32:23.758225 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:23.759831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:23.773138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 923/1814] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:23 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:23.960141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:25.051405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:25.387084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:25.388801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:25.405433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c98827-08e1-4651-a8a7-e7980cfe621a service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:25.410497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:25.411457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:25.411749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:25.412339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] PATH_INFO: `/v3/projects/e1b81510ee63469d91d223e6bd4afad8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:25.414516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1b81510ee63469d91d223e6bd4afad8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:25.416550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:25.543081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1b81510ee63469d91d223e6bd4afad8', 'request_id': 'req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:25.552188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1b81510ee63469d91d223e6bd4afad8', 'request_id': 'req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:25.561303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1b81510ee63469d91d223e6bd4afad8', 'request_id': 'req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:25.652144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] The token cache is being invalidate because project e1b81510ee63469d91d223e6bd4afad8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:25.652635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:25.653703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:25.658666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 892/1815] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:25 2022] DELETE /identity/v3/projects/e1b81510ee63469d91d223e6bd4afad8 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:25.659851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:25.675118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9094fb72-12f5-4eae-9c01-275b55ce5c82 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:25.741058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:25.742056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:25.742337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:25.742614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] PATH_INFO: `/v3/users/f5ec8d99b0494911ad6904a4dab44991` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:25.744074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f5ec8d99b0494911ad6904a4dab44991)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:25.746035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:26.037836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f5ec8d99b0494911ad6904a4dab44991'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:26.039152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f5ec8d99b0494911ad6904a4dab44991'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:26.039784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f5ec8d99b0494911ad6904a4dab44991'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:26.057051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f5ec8d99b0494911ad6904a4dab44991'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:26.065223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f5ec8d99b0494911ad6904a4dab44991'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:26.189432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 924/1816] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:25 2022] DELETE /identity/v3/users/f5ec8d99b0494911ad6904a4dab44991 => generated 0 bytes in 530 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:26.799690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:26.801412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:26.840452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:26.844102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:26.869993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c58f2e7-94a5-47e9-8cd2-74f5ce0a3436 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:26.932418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f12109-e4ee-4a9b-ad80-c2a2df5a0e42 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:26.933663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:26.934445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:26.934746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:26.935033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:26.937218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:26.938934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:26.940056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:26.940372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:26.940697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] PATH_INFO: `/v3/projects/8f70aced908a46528b4cf599e0344b40` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:26.942036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f70aced908a46528b4cf599e0344b40)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:26.944791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:27.026316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:27.028044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:27.028270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:27.041811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 893/1817] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:26 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:27.090214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f70aced908a46528b4cf599e0344b40', 'request_id': 'req-9d2f6086-a077-4f72-bed7-b543ae0889a4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.097809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f70aced908a46528b4cf599e0344b40', 'request_id': 'req-9d2f6086-a077-4f72-bed7-b543ae0889a4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.106877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f70aced908a46528b4cf599e0344b40', 'request_id': 'req-9d2f6086-a077-4f72-bed7-b543ae0889a4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.187886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] The token cache is being invalidate because project 8f70aced908a46528b4cf599e0344b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:27.188363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.189134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.191393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 925/1818] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:26 2022] DELETE /identity/v3/projects/8f70aced908a46528b4cf599e0344b40 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:27.197829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:27.216274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ae363d-d4c5-4a63-8050-c4803933bb82 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:27.292731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:27.292731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:27.293079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:27.293435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] PATH_INFO: `/v3/users/fa20ce99edec4485bd644e1225f76548` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:27.295416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa20ce99edec4485bd644e1225f76548)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:27.297834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:27.561697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa20ce99edec4485bd644e1225f76548'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.577158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa20ce99edec4485bd644e1225f76548'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.598855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa20ce99edec4485bd644e1225f76548'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.599695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa20ce99edec4485bd644e1225f76548'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.763349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bde98c13-4d51-4aa2-b1b1-cabade173d39 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa20ce99edec4485bd644e1225f76548'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:27.765169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 894/1819] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:27 2022] DELETE /identity/v3/users/fa20ce99edec4485bd644e1225f76548 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:28.016453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:28.293725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:28.295245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:28.312389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f6086-a077-4f72-bed7-b543ae0889a4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:28.377779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:28.378658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:28.378943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:28.379242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] PATH_INFO: `/v3/projects/feb2ee4c4de34ef0b214639fb2226ed0` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:28.380461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb2ee4c4de34ef0b214639fb2226ed0)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:28.382838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:28.541258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb2ee4c4de34ef0b214639fb2226ed0', 'request_id': 'req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:28.547887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb2ee4c4de34ef0b214639fb2226ed0', 'request_id': 'req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:28.555495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb2ee4c4de34ef0b214639fb2226ed0', 'request_id': 'req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:28.668337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] The token cache is being invalidate because project feb2ee4c4de34ef0b214639fb2226ed0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:28.668678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:28.673100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fd7c38d2-a11a-4b2e-b6aa-7db6bdeefdb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:28.673100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 926/1820] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:28 2022] DELETE /identity/v3/projects/feb2ee4c4de34ef0b214639fb2226ed0 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:28.677201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:28.677508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:28.677877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:28.700759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:28.702008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:28.781278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 895/1821] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:28.791075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:28.791362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:28.791644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:28.813108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:28.814465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:28.846438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 927/1822] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:28.851461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:28.861887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60db85f-bafd-484f-a9a6-413a08d7b216 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:28.869264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:28.870216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:28.870654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:28.871071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:28.872008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:28.872517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:28.876850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:28.893062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 896/1823] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:28.894313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:28.904274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7258e59-a87f-4095-96c8-2701834f565c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:28.908955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:28.909703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:28.910333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:28.910754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:28.911600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:28.914200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:29.056056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 928/1824] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:28 2022] POST /identity/v3/projects => generated 375 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:29.057282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:29.069997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b85fa7-c51c-417f-8d48-18d8e16162ab admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:29.149094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:29.151312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:29.151312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:29.151312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:29.157758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:29.160807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:29.327269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 897/1825] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:29 2022] POST /identity/v3/users => generated 288 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:29.331218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:29.343046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db922797-d1ad-42e6-a676-e6c592bca22d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:29.347689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:29.348360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:29.348687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:29.348969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:29.350576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:29.352187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:29.362880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 929/1826] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:29.371733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:29.382466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b730276d-910a-49a7-8351-a4102aea847f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:29.387307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8daf2158-d78a-48fc-a299-706c3c73a188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:29.387992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2158-d78a-48fc-a299-706c3c73a188 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:29.388281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2158-d78a-48fc-a299-706c3c73a188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:29.388586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8daf2158-d78a-48fc-a299-706c3c73a188 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:29.389343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf2158-d78a-48fc-a299-706c3c73a188 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:29.391102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8daf2158-d78a-48fc-a299-706c3c73a188 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:29.402941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 898/1827] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:29 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:29.407855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:29.418766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31407f16-f16b-4c97-8924-7e7c69bb0ec5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:29.436325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:29.436325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:29.436325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:29.436325 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] PATH_INFO: `/v3/projects/51422bbc67de49a680e627427cb5ef47/users/774570ab95784bbdb435b37ae435fa03/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:29.439747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] RBAC: Authorizing `identity:create_grant(project_id=51422bbc67de49a680e627427cb5ef47, user_id=774570ab95784bbdb435b37ae435fa03, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:29.442473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:29.638164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 930/1828] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:29 2022] PUT /identity/v3/projects/51422bbc67de49a680e627427cb5ef47/users/774570ab95784bbdb435b37ae435fa03/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:29.651269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:29.651564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:29.651850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:29.685768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] MFA Rules not processed for user `774570ab95784bbdb435b37ae435fa03`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:29.687220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:29.767881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 899/1829] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:29 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:29.781182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:29.783230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:29.817415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:29.881361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0bec12e-5bbe-4629-88ba-1b9799eb0149 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:29.887651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03c0588-cd03-460e-bdf4-80e7b0e8bef1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:29.889084 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0588-cd03-460e-bdf4-80e7b0e8bef1 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:29.889482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0588-cd03-460e-bdf4-80e7b0e8bef1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:29.889865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03c0588-cd03-460e-bdf4-80e7b0e8bef1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:29.903380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c0588-cd03-460e-bdf4-80e7b0e8bef1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:29.905273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03c0588-cd03-460e-bdf4-80e7b0e8bef1 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:29.919854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 931/1830] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:29 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:30.734542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:31.965606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:32.576523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:33.528840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:33.977458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:35.159834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:35.828493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:35.984882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:36.101154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49cc0e-b529-421e-837f-43aa391d7d4d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:36.107058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:36.107978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:36.108261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:36.108535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] PATH_INFO: `/v3/users/774570ab95784bbdb435b37ae435fa03` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:36.109844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] RBAC: Authorizing `identity:delete_user(user_id=774570ab95784bbdb435b37ae435fa03)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:36.112961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:36.446411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '774570ab95784bbdb435b37ae435fa03'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:36.453748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '774570ab95784bbdb435b37ae435fa03'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:36.468039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '774570ab95784bbdb435b37ae435fa03'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:36.469121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '774570ab95784bbdb435b37ae435fa03'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:36.545789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '774570ab95784bbdb435b37ae435fa03'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:36.552354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 900/1831] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:35 2022] DELETE /identity/v3/users/774570ab95784bbdb435b37ae435fa03 => generated 0 bytes in 568 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:36.558109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:36.558616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:36.558755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:36.588543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:36.588543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:36.680198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 932/1832] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:36.690812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:36.692491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:36.773612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c08318b-98b9-4c11-a840-5c8706d4c340 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:36.777407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d69cfaf2-31b3-4601-9c7d-48d4b000671f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:36.778063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cfaf2-31b3-4601-9c7d-48d4b000671f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:36.778348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cfaf2-31b3-4601-9c7d-48d4b000671f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:36.778623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d69cfaf2-31b3-4601-9c7d-48d4b000671f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:36.788365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cfaf2-31b3-4601-9c7d-48d4b000671f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:36.790005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d69cfaf2-31b3-4601-9c7d-48d4b000671f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:36.800648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 901/1833] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:37.359182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:37.362144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:37.392734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98ca786-b7f2-4735-b8c1-16f93b8a524d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:37.402552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:37.404179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:37.404729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:37.405474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] PATH_INFO: `/v3/projects/51422bbc67de49a680e627427cb5ef47` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:37.407892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=51422bbc67de49a680e627427cb5ef47)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:37.413476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:37.590085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '51422bbc67de49a680e627427cb5ef47', 'request_id': 'req-2e9bde43-5787-4981-b935-38ed9a4b23f7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:37.596443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '51422bbc67de49a680e627427cb5ef47', 'request_id': 'req-2e9bde43-5787-4981-b935-38ed9a4b23f7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:37.602327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '51422bbc67de49a680e627427cb5ef47', 'request_id': 'req-2e9bde43-5787-4981-b935-38ed9a4b23f7'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:37.688074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] The token cache is being invalidate because project 51422bbc67de49a680e627427cb5ef47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:37.688372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:37.689042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2e9bde43-5787-4981-b935-38ed9a4b23f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:37.690616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 933/1834] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:37 2022] DELETE /identity/v3/projects/51422bbc67de49a680e627427cb5ef47 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:37.696531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:37.697056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:37.697158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:37.715983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:37.717249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:37.789556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 902/1835] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:37.792044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:37.792886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:37.793690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:37.800904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:37.821418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:37.822715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:37.834696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 934/1836] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:37.838606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:37.857319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0f1153-a9be-4ee6-aa40-fcb0c75c9269 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:37.864219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:37.866537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:37.867056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:37.867486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:37.868752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:37.869237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:37.871100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:37.894614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 903/1837] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:37.899296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:37.915703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aa1cc5c-d9ab-4f91-9d83-dba6dcd5b4aa None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:37.922493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:37.923757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:37.924195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:37.924653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:37.925586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:37.928489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:38.078075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:38.085656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 935/1838] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:37 2022] POST /identity/v3/projects => generated 369 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:38.090247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:38.102428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3ac3e-ffa7-4440-ab84-264ecd95a242 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:38.165539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:38.166182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:38.166480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:38.170337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:38.170337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:38.170337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:38.305627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 904/1839] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:38 2022] POST /identity/v3/users => generated 285 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:38.309818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:38.330377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ccd588-87ae-4b7f-bfc2-6c912aef203b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:38.339369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:38.340348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:38.340682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:38.340968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:38.341731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:38.343385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:38.358625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 936/1840] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:38.362955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:38.375235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd5ca58-a7c4-4b82-a747-316c7c9e1cd8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:38.380709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa093836-401f-4756-80dd-45334935c7d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:38.383004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093836-401f-4756-80dd-45334935c7d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:38.383303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093836-401f-4756-80dd-45334935c7d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:38.383586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa093836-401f-4756-80dd-45334935c7d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:38.384618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093836-401f-4756-80dd-45334935c7d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:38.386730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa093836-401f-4756-80dd-45334935c7d5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:38.400439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 905/1841] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:38 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:38.419643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:38.434424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb031190-c8a8-468a-8763-f00a0b8a8f9b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:38.442293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:38.444603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:38.445125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:38.446470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] PATH_INFO: `/v3/projects/9b1bce8c77b6416d972e8e935d692663/users/b6cd7cf8f80749b9ab449776f169ce2a/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:38.473268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b1bce8c77b6416d972e8e935d692663, user_id=b6cd7cf8f80749b9ab449776f169ce2a, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:38.473268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:38.571142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 937/1842] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:38 2022] PUT /identity/v3/projects/9b1bce8c77b6416d972e8e935d692663/users/b6cd7cf8f80749b9ab449776f169ce2a/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:38.582499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:38.582802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:38.583690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:38.668687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] MFA Rules not processed for user `b6cd7cf8f80749b9ab449776f169ce2a`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:38.675478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:38.829995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 906/1843] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:38 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:38.862056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:38.864227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:38.976992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e9e305-ec65-49cb-803e-f3ecf2fa6d8b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:38.986061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca74a07-9e82-450a-8e8b-ce43f36ac599 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:38.988012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74a07-9e82-450a-8e8b-ce43f36ac599 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:38.990602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74a07-9e82-450a-8e8b-ce43f36ac599 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:38.994509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca74a07-9e82-450a-8e8b-ce43f36ac599 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:39.032649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74a07-9e82-450a-8e8b-ce43f36ac599 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:39.041919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca74a07-9e82-450a-8e8b-ce43f36ac599 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:39.075189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 938/1844] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:38 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:41.074386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:41.580300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:42.899057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:42.903835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:42.987154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3236377e-4d46-4c3e-a98f-5307f4cfe637 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:42.992506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:42.993674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:42.993954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:42.994227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] PATH_INFO: `/v3/users/f403119bf4dc4423be4fc15c4badcc6c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:42.995420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] RBAC: Authorizing `identity:delete_user(user_id=f403119bf4dc4423be4fc15c4badcc6c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:42.997408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:43.214431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f403119bf4dc4423be4fc15c4badcc6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:43.221751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f403119bf4dc4423be4fc15c4badcc6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:43.235078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f403119bf4dc4423be4fc15c4badcc6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:43.235858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f403119bf4dc4423be4fc15c4badcc6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:43.305996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f403119bf4dc4423be4fc15c4badcc6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:43.316624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 907/1845] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:42 2022] DELETE /identity/v3/users/f403119bf4dc4423be4fc15c4badcc6c => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:43.322293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:43.322906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:43.322906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:43.341415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:43.343005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:43.433775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 939/1846] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:43.451354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:43.452908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:43.554605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89eb22e-fc6a-40bb-9897-9ac3af78e5df admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:43.559004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:43.559004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:43.559004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:43.560793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:43.585287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:43.586804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:43.601366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 908/1847] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:43.674455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:44.651558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:44.664673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:44.666160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089118fc-1f16-4bf1-8cd7-06e230b5f15d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:44.673285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:44.674456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:44.674959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:44.675473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] PATH_INFO: `/v3/projects/44e40c5dd9a3441189c3cbd17c7b45ae` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:44.677023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] RBAC: Authorizing `identity:delete_project(project_id=44e40c5dd9a3441189c3cbd17c7b45ae)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:44.680765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:44.935915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:44.941236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44e40c5dd9a3441189c3cbd17c7b45ae', 'request_id': 'req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:44.948662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44e40c5dd9a3441189c3cbd17c7b45ae', 'request_id': 'req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:44.954767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44e40c5dd9a3441189c3cbd17c7b45ae', 'request_id': 'req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.071405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] The token cache is being invalidate because project 44e40c5dd9a3441189c3cbd17c7b45ae was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:45.071823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.072606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.075114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 940/1848] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:44 2022] DELETE /identity/v3/projects/44e40c5dd9a3441189c3cbd17c7b45ae => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:45.078870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:45.090316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec5324-c1e7-456a-adf0-4d8264183092 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:45.180474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:45.180474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:45.180474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:45.180474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] PATH_INFO: `/v3/users/8048bccaba5042aebe0e8bff99a9a7b5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:45.180797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8048bccaba5042aebe0e8bff99a9a7b5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:45.180797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:45.485606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8048bccaba5042aebe0e8bff99a9a7b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.494774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8048bccaba5042aebe0e8bff99a9a7b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.509533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8048bccaba5042aebe0e8bff99a9a7b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.511477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8048bccaba5042aebe0e8bff99a9a7b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.609352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ac95ad1b-8d17-4414-a691-e2539ff4c402 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8048bccaba5042aebe0e8bff99a9a7b5'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:45.618607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 909/1849] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:45 2022] DELETE /identity/v3/users/8048bccaba5042aebe0e8bff99a9a7b5 => generated 0 bytes in 540 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:46.290461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:46.292080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:46.317312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fdcfef6-df77-4de6-b0f5-0289cdbb351c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:46.440043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:46.441718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:46.442173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:46.443012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] PATH_INFO: `/v3/projects/eaa4ae14a9bd47eba313542b4a488ce4` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:46.446023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eaa4ae14a9bd47eba313542b4a488ce4)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:46.448693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:46.643612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eaa4ae14a9bd47eba313542b4a488ce4', 'request_id': 'req-49127b0d-74f0-4df9-9d66-208b6f6241c4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:46.654614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eaa4ae14a9bd47eba313542b4a488ce4', 'request_id': 'req-49127b0d-74f0-4df9-9d66-208b6f6241c4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:46.662411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eaa4ae14a9bd47eba313542b4a488ce4', 'request_id': 'req-49127b0d-74f0-4df9-9d66-208b6f6241c4'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:46.773486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] The token cache is being invalidate because project eaa4ae14a9bd47eba313542b4a488ce4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:46.773785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:46.774466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-49127b0d-74f0-4df9-9d66-208b6f6241c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:46.776082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 941/1850] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:46 2022] DELETE /identity/v3/projects/eaa4ae14a9bd47eba313542b4a488ce4 => generated 0 bytes in 486 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:46.785322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:46.785322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:46.785322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:46.809175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:46.810470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:46.879291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 910/1851] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:46.887342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:46.887633 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:46.891989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:46.907777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:46.908767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:46.917539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 942/1852] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:46.922053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:46.932852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc78a8-3ae2-4b82-9924-60f7f30a51fe None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:46.938219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:46.939028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:46.939180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:46.939466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:46.940403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:46.940604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:46.943475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:46.961997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 911/1853] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:46.963200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:46.974411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57402cd9-45c0-469c-bdbe-30f31838af91 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:46.980304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:46.981135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:46.981490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:46.981805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:46.982611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:46.985181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:47.090302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 943/1854] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:46 2022] POST /identity/v3/projects => generated 381 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:47.091549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:47.102436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276939b5-5b0d-4619-bfa4-65ed9d9fd135 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:47.165269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:47.165978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.166270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.166547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.167259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:47.169266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:47.253561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 912/1855] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:47 2022] POST /identity/v3/users => generated 291 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:47.258370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:47.269360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b2cf7-54a3-4322-8634-18f9774f09bd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:47.275109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:47.275897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.276226 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.276593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.277572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:47.280342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:47.281136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:47.293718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 944/1856] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:47.297938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:47.308948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a04c9c-1b16-40fd-9317-6ef3dc7426ce admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:47.313623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa38df45-db20-4e1a-9df5-9b8bff5ca31c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:47.314291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38df45-db20-4e1a-9df5-9b8bff5ca31c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.314570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38df45-db20-4e1a-9df5-9b8bff5ca31c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.314843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa38df45-db20-4e1a-9df5-9b8bff5ca31c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.315574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38df45-db20-4e1a-9df5-9b8bff5ca31c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:47.317219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa38df45-db20-4e1a-9df5-9b8bff5ca31c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:47.330804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 913/1857] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:47 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:47.332105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:47.356017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baa5d8d-46a7-49f0-ae95-e7dbef96aa8a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:47.361012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:47.361920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.362208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.365943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] PATH_INFO: `/v3/projects/ac8eed93f88f4983b9ecb66cce6717c5/users/1207469b6f6744b792b0baf4036517da/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.379962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac8eed93f88f4983b9ecb66cce6717c5, user_id=1207469b6f6744b792b0baf4036517da, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:47.383674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:47.464858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 945/1858] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:47 2022] PUT /identity/v3/projects/ac8eed93f88f4983b9ecb66cce6717c5/users/1207469b6f6744b792b0baf4036517da/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:47.477149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.479266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.479438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.515815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] MFA Rules not processed for user `1207469b6f6744b792b0baf4036517da`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:47.517165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:47.611533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 914/1859] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:47.611756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:47.614198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:47.684961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be29d8a-9294-4d9e-883a-03c12b6c5af0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:47.689673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:47.690592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.690948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.691178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.691882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:47.693870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:47.825483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 946/1860] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:47 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:47.829386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:47.842082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb783f8b-caf1-46cd-b893-40718e9c8f34 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:47.909701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:47.910667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:47.911100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:47.911559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:47.917342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:47.920201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:48.058995 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 915/1861] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:47 2022] POST /identity/v3/users => generated 297 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:48.062718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:48.074890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970bfa8-f06f-4335-87ad-ea02509252fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:48.085346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:48.085713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:48.085993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:48.086268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:48.088193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:48.089859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:48.104220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 947/1862] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:48.109900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:48.123251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad4582f-ea3d-42a0-9b64-7f6c15c499d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:48.128225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:48.129438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:48.129903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:48.140400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] PATH_INFO: `/v3/projects/97dbac0d44fe409fb3bcc6d598374c4b/users/af844719d5964b43b9c5c152ba979008/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:48.159693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dbac0d44fe409fb3bcc6d598374c4b, user_id=af844719d5964b43b9c5c152ba979008, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:48.168361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:48.300741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 916/1863] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:48 2022] PUT /identity/v3/projects/97dbac0d44fe409fb3bcc6d598374c4b/users/af844719d5964b43b9c5c152ba979008/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:48.305025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:48.317888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cddc8614-9b4d-46cf-9ff1-4eec6e4225a1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:48.390732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f0409d-6ba2-4ff8-8f39-28263f17d4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:48.391700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0409d-6ba2-4ff8-8f39-28263f17d4c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:48.392116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0409d-6ba2-4ff8-8f39-28263f17d4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:48.392472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f0409d-6ba2-4ff8-8f39-28263f17d4c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:48.393894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0409d-6ba2-4ff8-8f39-28263f17d4c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:48.395045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f0409d-6ba2-4ff8-8f39-28263f17d4c4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:48.414417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 948/1864] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:48 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:48.417974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:48.430597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-878b4e88-ccaa-46c0-a802-0abd9697be5d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:48.446664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:48.448206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:48.448206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:48.448206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] PATH_INFO: `/v3/projects/97dbac0d44fe409fb3bcc6d598374c4b/users/af844719d5964b43b9c5c152ba979008/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:48.450573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] RBAC: Authorizing `identity:create_grant(project_id=97dbac0d44fe409fb3bcc6d598374c4b, user_id=af844719d5964b43b9c5c152ba979008, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:48.453405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:48.465676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 917/1865] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:48 2022] PUT /identity/v3/projects/97dbac0d44fe409fb3bcc6d598374c4b/users/af844719d5964b43b9c5c152ba979008/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:48.478272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:48.478559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:48.478861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:48.534200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] MFA Rules not processed for user `af844719d5964b43b9c5c152ba979008`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:48.535593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:48.618089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 949/1866] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:48 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:48.634485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:48.634485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:48.728102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a56c59f-13b7-4fde-ae10-afd9271f435d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:48.735570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:48.736256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:48.736606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:48.736910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:48.751643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:48.753573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:48.768259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 918/1867] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:48 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:32:49.975312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:50.114333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:50.124675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9916dc64-c32f-4663-a6fe-5aec10d1e274 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:50.129551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af844719d5964b43b9c5c152ba979008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dbac0d44fe409fb3bcc6d598374c4b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:50.130221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:50.130518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:50.131464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:50.132205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:50.132629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:50.134167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:50.148598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 950/1868] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:50.152867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:50.163221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64405099-de45-4068-9ae6-32c008798eb3 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:50.167856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af844719d5964b43b9c5c152ba979008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dbac0d44fe409fb3bcc6d598374c4b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:50.168743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:50.168871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:50.169130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:50.169798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:50.171699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:50.254430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:50.260844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:50.286216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 919/1869] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:50 2022] POST /identity/v3/projects => generated 354 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:50.290392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:50.350712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f643b08a-268a-4b75-b6c7-b376c8740f8e tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:50.374519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78b76ce-89ac-4688-a3e6-867918221205 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:50.379327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af844719d5964b43b9c5c152ba979008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dbac0d44fe409fb3bcc6d598374c4b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:50.380002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:50.380299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:50.380675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:50.381605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:50.381927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:50.383566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:50.398136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 920/1870] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:50.417551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:50.418424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:50.418705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:50.418990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] PATH_INFO: `/v3/users/d877c29d385d4d5fac2b07efdc40197f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:50.420211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d877c29d385d4d5fac2b07efdc40197f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:50.422185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:50.437607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:50.447848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ec4c39-d39f-4959-b8bf-5d30a5a814cf tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:50.453775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'af844719d5964b43b9c5c152ba979008', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '97dbac0d44fe409fb3bcc6d598374c4b', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:50.454083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:50.454202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:50.454393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] PATH_INFO: `/v3/projects/7b37ca2afb594131a823018174cd3fe9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:50.455594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorizing `identity:delete_project(project_id=7b37ca2afb594131a823018174cd3fe9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:50.457576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:50.717698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd877c29d385d4d5fac2b07efdc40197f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.718393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd877c29d385d4d5fac2b07efdc40197f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.719311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd877c29d385d4d5fac2b07efdc40197f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.729588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b37ca2afb594131a823018174cd3fe9', 'request_id': 'req-a3d69583-f06a-43ef-a1e2-14851c064b73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.736302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd877c29d385d4d5fac2b07efdc40197f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.737739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b37ca2afb594131a823018174cd3fe9', 'request_id': 'req-a3d69583-f06a-43ef-a1e2-14851c064b73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.745184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b37ca2afb594131a823018174cd3fe9', 'request_id': 'req-a3d69583-f06a-43ef-a1e2-14851c064b73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.745417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd877c29d385d4d5fac2b07efdc40197f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.834398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:50.871624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] The token cache is being invalidate because project 7b37ca2afb594131a823018174cd3fe9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:50.872003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.872740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:50.874673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 921/1871] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:50 2022] DELETE /identity/v3/projects/7b37ca2afb594131a823018174cd3fe9 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:51.029460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 951/1872] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:50 2022] DELETE /identity/v3/users/d877c29d385d4d5fac2b07efdc40197f => generated 0 bytes in 775 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:51.313353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:51.315360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:51.409064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d69583-f06a-43ef-a1e2-14851c064b73 tempest-VolumeQuotaClassesTest-1424921214 tempest-VolumeQuotaClassesTest-1424921214-project-admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:51.478441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:51.479335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:51.480013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:51.480013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] PATH_INFO: `/v3/projects/a3f1f93169544b91b4d05f9af07a0f61` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:51.481233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a3f1f93169544b91b4d05f9af07a0f61)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:51.483192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:51.613036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a3f1f93169544b91b4d05f9af07a0f61', 'request_id': 'req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:51.620875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a3f1f93169544b91b4d05f9af07a0f61', 'request_id': 'req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:51.628992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a3f1f93169544b91b4d05f9af07a0f61', 'request_id': 'req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:51.695858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:51.699809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:51.715683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066ffa21-11b9-475b-8176-5ba39e5a1333 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:51.764735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] The token cache is being invalidate because project a3f1f93169544b91b4d05f9af07a0f61 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:51.765061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:51.765739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:51.767421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 922/1873] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:51 2022] DELETE /identity/v3/projects/a3f1f93169544b91b4d05f9af07a0f61 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:51.772479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:51.774804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:51.791311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:51.797295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:51.797295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:51.797295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] PATH_INFO: `/v3/users/1207469b6f6744b792b0baf4036517da` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:51.797295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1207469b6f6744b792b0baf4036517da)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:51.797295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:51.847597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e889a43-8d08-4a0e-bcf4-8259cfe6b488 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:51.852893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:51.853783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:51.854067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:51.854345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] PATH_INFO: `/v3/users/cba526554d8149ebb6345e988c539c04` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:51.855802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] RBAC: Authorizing `identity:delete_user(user_id=cba526554d8149ebb6345e988c539c04)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:51.857805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:51.986244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:52.001991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1207469b6f6744b792b0baf4036517da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.002776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1207469b6f6744b792b0baf4036517da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.003459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1207469b6f6744b792b0baf4036517da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.017312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1207469b6f6744b792b0baf4036517da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.025307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-97e78f0c-279b-454f-9e75-474a1d804236 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1207469b6f6744b792b0baf4036517da'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.083695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cba526554d8149ebb6345e988c539c04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.093222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cba526554d8149ebb6345e988c539c04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.107049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cba526554d8149ebb6345e988c539c04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.107873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cba526554d8149ebb6345e988c539c04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.117038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 952/1874] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:51 2022] DELETE /identity/v3/users/1207469b6f6744b792b0baf4036517da => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:52.121855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:52.122213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:52.122564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:52.142828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:52.144244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:52.225962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 953/1875] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:52.228409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cba526554d8149ebb6345e988c539c04'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.235003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:52.236697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:52.237237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 923/1876] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:51 2022] DELETE /identity/v3/users/cba526554d8149ebb6345e988c539c04 => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:52.330097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bae7d94-35ae-4835-8bda-58b32bc7144b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:52.400323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:52.401044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:52.401345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:52.401611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:52.404349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:52.482335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:52.484214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21beaf58-bd7a-476a-a983-37f6a1fb5e4f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:52.500856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 954/1877] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:52.576093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:52.577699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:52.602630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac3a9ea-667b-4e73-b102-cb37d1d66cee admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:52.608705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:52.609816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:52.621690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:52.622203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] PATH_INFO: `/v3/projects/9985fe88562c4e33a10a53be41fc7364` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:52.624000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9985fe88562c4e33a10a53be41fc7364)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:52.627194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:52.826740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9985fe88562c4e33a10a53be41fc7364', 'request_id': 'req-74b2c8c2-0429-4850-85bf-9120629dd904'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.833497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9985fe88562c4e33a10a53be41fc7364', 'request_id': 'req-74b2c8c2-0429-4850-85bf-9120629dd904'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.840325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9985fe88562c4e33a10a53be41fc7364', 'request_id': 'req-74b2c8c2-0429-4850-85bf-9120629dd904'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.919791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] The token cache is being invalidate because project 9985fe88562c4e33a10a53be41fc7364 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:52.920094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.922137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-74b2c8c2-0429-4850-85bf-9120629dd904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:52.924976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 924/1878] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:52 2022] DELETE /identity/v3/projects/9985fe88562c4e33a10a53be41fc7364 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:52.930525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:52.933881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:52.934463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:52.974184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:52.978776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:52.982229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:53.095258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 955/1879] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:53.108811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.109110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.109391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.128877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:53.130037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:53.138538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 925/1880] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:53.151030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.161110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb9fbb2-81d9-4a52-a073-d87d9d7ea034 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.166615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.167295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.167581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.167853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.168601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:53.168912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.170510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.184740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 956/1881] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:53.225399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.226549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.230176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:53.242268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53e4c16-7d44-4ee9-a313-43e7b4f0a9bf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.246410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36760aaa-76e3-4d23-87f5-cb1e80a26ed7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.247265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.248070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.248435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.248867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.249654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.251393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.252094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.252259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.252553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.252873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] PATH_INFO: `/v3/projects/ac8eed93f88f4983b9ecb66cce6717c5` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.254099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac8eed93f88f4983b9ecb66cce6717c5)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.256793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.395987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 957/1882] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:53 2022] POST /identity/v3/projects => generated 373 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:53.399489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.424884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac8eed93f88f4983b9ecb66cce6717c5', 'request_id': 'req-3dac8f58-2624-4662-addd-ca2a9f5c9327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:53.444951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac8eed93f88f4983b9ecb66cce6717c5', 'request_id': 'req-3dac8f58-2624-4662-addd-ca2a9f5c9327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:53.468611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac8eed93f88f4983b9ecb66cce6717c5', 'request_id': 'req-3dac8f58-2624-4662-addd-ca2a9f5c9327'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:53.560916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc68d81-47a6-44d2-9f47-102b5a441d2d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.569165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.570077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.570590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.570946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.572242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.574866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.584835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] The token cache is being invalidate because project ac8eed93f88f4983b9ecb66cce6717c5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:53.585577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:53.586292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:53.588391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 926/1883] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:53 2022] DELETE /identity/v3/projects/ac8eed93f88f4983b9ecb66cce6717c5 => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:53.597818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.609737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:53.768023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 958/1884] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:53 2022] POST /identity/v3/users => generated 287 bytes in 371 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:53.771004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dac8f58-2624-4662-addd-ca2a9f5c9327 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.772862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.775028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:53.779197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.780265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.780641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.780989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] PATH_INFO: `/v3/users/af844719d5964b43b9c5c152ba979008` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.782347 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af844719d5964b43b9c5c152ba979008)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.785572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.797123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fde1c8-ce5f-45f2-80b2-0252da1fec84 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.804329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.805068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.805361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.805642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.806518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.808031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.823273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 959/1885] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:53.834789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.851442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e04e413-5c36-4af8-8bdb-dd3f6ba98c3b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.858729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.861838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.861838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.861838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.861838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.862358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:53.876151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 960/1886] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:53.882592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:53.894625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84af8a8-d3a3-4ad4-96e1-a83c6d906323 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:53.900490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1014ae-a385-4dc8-85e5-7561bd9ca13f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:53.901639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1014ae-a385-4dc8-85e5-7561bd9ca13f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:53.901928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1014ae-a385-4dc8-85e5-7561bd9ca13f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:53.902210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1014ae-a385-4dc8-85e5-7561bd9ca13f admin admin] PATH_INFO: `/v3/projects/7c9437720792445e9acc0b877b4d7890/users/e6c0911f4ddc4bec8ed498d5eb35a74b/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:53.937361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1014ae-a385-4dc8-85e5-7561bd9ca13f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c9437720792445e9acc0b877b4d7890, user_id=e6c0911f4ddc4bec8ed498d5eb35a74b, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:53.941097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1014ae-a385-4dc8-85e5-7561bd9ca13f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:54.057912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 961/1887] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:53 2022] PUT /identity/v3/projects/7c9437720792445e9acc0b877b4d7890/users/e6c0911f4ddc4bec8ed498d5eb35a74b/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:54.070567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:54.071002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:54.071295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:54.092640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af844719d5964b43b9c5c152ba979008'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.110252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af844719d5964b43b9c5c152ba979008'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.111552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] MFA Rules not processed for user `e6c0911f4ddc4bec8ed498d5eb35a74b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:54.113728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:54.124549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af844719d5964b43b9c5c152ba979008'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.125614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af844719d5964b43b9c5c152ba979008'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.195234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af844719d5964b43b9c5c152ba979008'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.197521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 962/1888] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:54.202219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 927/1889] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:53 2022] DELETE /identity/v3/users/af844719d5964b43b9c5c152ba979008 => generated 0 bytes in 607 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:54.209205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:54.210891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:54.284772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9fc2f-3178-4a2f-86b3-1141f7feab56 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:54.351168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:54.351832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:54.352122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:54.352428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:54.354273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:54.420985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:54.422451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:54.433073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 963/1890] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:54 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:32:54.460388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:54.460388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:54.529997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefee03f-be7a-4d2f-8f39-ce151f381ac3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:54.535037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:54.535923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:54.537959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:54.537959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] PATH_INFO: `/v3/users/e6c0911f4ddc4bec8ed498d5eb35a74b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:54.538712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6c0911f4ddc4bec8ed498d5eb35a74b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:54.540721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:54.889636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:54.891205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:54.908502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17cde121-fcff-4dc1-b937-f5d36f52ee1a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:54.914004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:54.915055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:54.915349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:54.917323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] PATH_INFO: `/v3/projects/97dbac0d44fe409fb3bcc6d598374c4b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:54.917323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] RBAC: Authorizing `identity:delete_project(project_id=97dbac0d44fe409fb3bcc6d598374c4b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:54.931040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:54.952760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6c0911f4ddc4bec8ed498d5eb35a74b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.966729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6c0911f4ddc4bec8ed498d5eb35a74b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.990068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6c0911f4ddc4bec8ed498d5eb35a74b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:54.992018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6c0911f4ddc4bec8ed498d5eb35a74b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.130246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97dbac0d44fe409fb3bcc6d598374c4b', 'request_id': 'req-5caf857b-6690-44e9-a7a0-6cf593f2946a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.137785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97dbac0d44fe409fb3bcc6d598374c4b', 'request_id': 'req-5caf857b-6690-44e9-a7a0-6cf593f2946a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.143467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97dbac0d44fe409fb3bcc6d598374c4b', 'request_id': 'req-5caf857b-6690-44e9-a7a0-6cf593f2946a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.166793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-41aac532-7a25-40ea-a6e3-3051b53ed3f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6c0911f4ddc4bec8ed498d5eb35a74b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.174121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 928/1891] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:32:54 2022] DELETE /identity/v3/users/e6c0911f4ddc4bec8ed498d5eb35a74b => generated 0 bytes in 721 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:55.178442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.178732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.179008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.203709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:55.207434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:55.265367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] The token cache is being invalidate because project 97dbac0d44fe409fb3bcc6d598374c4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:55.265825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.266646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5caf857b-6690-44e9-a7a0-6cf593f2946a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:55.269835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 964/1892] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:54 2022] DELETE /identity/v3/projects/97dbac0d44fe409fb3bcc6d598374c4b => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:55.276699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b9bd-dab7-4873-aa56-ec56027763d3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.276992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b9bd-dab7-4873-aa56-ec56027763d3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.277378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a5b9bd-dab7-4873-aa56-ec56027763d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.281882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 929/1893] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:55.293795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.295296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:55.299589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-37a5b9bd-dab7-4873-aa56-ec56027763d3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:55.301117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-37a5b9bd-dab7-4873-aa56-ec56027763d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:55.309682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 965/1894] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:55.323472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.323758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.324053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.336892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:55.344528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:55.345910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:55.355532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 966/1895] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:55.359979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.361777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a504da66-1f1a-44ff-a0d8-ea052f040a23 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.366016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.374198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.374198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.374198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.377339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f55029-9c3a-44b2-b097-8583eb6cebb3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.382906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.383583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.383866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.384160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.385300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:55.385300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:55.386829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:55.387705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:55.388693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:55.401030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 967/1897] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:32:55.404612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 930/1897] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:32:55.405931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.418839 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385165cd-e0af-434e-a09f-7d1d044aab2b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.425873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.426776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.427166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.427719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.428811 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:55.438246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:55.594991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 968/1898] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:55 2022] POST /identity/v3/projects => generated 387 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:55.596046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.641035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68387dc5-2aab-47f6-a186-d34167fdfc37 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.722021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.722816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.725323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.727792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.729042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:55.731779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:55.856949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 931/1899] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:55 2022] POST /identity/v3/users => generated 294 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:55.861211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.898032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712410ba-983c-41fc-b42b-afcf6507e93d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.902733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.903410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.903694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.903973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.904738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:55.906337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:55.917329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 969/1900] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:55.921634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.932325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dbf8a5-ea50-4876-8dc9-146a39ba3d10 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.937365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.938065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.938352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.938670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.939411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:55.941071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:55.953320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 932/1901] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:55.957597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.968272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75946a81-48d6-4a7e-be2d-f55073cab1dd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:55.971949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:55.973732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:55.979163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733d931c-7065-494f-83a4-d5db0de7cb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:55.980031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d931c-7065-494f-83a4-d5db0de7cb0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:55.980316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d931c-7065-494f-83a4-d5db0de7cb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:55.980622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733d931c-7065-494f-83a4-d5db0de7cb0f admin admin] PATH_INFO: `/v3/projects/8ebd38a39c064f91b80fd795ee7f8801/users/912f6c1468884b25a5fd4ddb2677378c/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:55.998945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:56.006918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89c3d495-a3ea-48ab-b7cd-9c17298dcbb7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.007668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d931c-7065-494f-83a4-d5db0de7cb0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ebd38a39c064f91b80fd795ee7f8801, user_id=912f6c1468884b25a5fd4ddb2677378c, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.014198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733d931c-7065-494f-83a4-d5db0de7cb0f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.014390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.015281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.015640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.015990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] PATH_INFO: `/v3/projects/7c9437720792445e9acc0b877b4d7890` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.017732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c9437720792445e9acc0b877b4d7890)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.019892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.102499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 970/1902] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:55 2022] PUT /identity/v3/projects/8ebd38a39c064f91b80fd795ee7f8801/users/912f6c1468884b25a5fd4ddb2677378c/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:56.114293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.114605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.114789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.146699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] MFA Rules not processed for user `912f6c1468884b25a5fd4ddb2677378c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:56.147813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:56.155278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c9437720792445e9acc0b877b4d7890', 'request_id': 'req-7941cbad-b745-4b1f-9071-b315d0cda75c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:56.167612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c9437720792445e9acc0b877b4d7890', 'request_id': 'req-7941cbad-b745-4b1f-9071-b315d0cda75c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:56.187285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c9437720792445e9acc0b877b4d7890', 'request_id': 'req-7941cbad-b745-4b1f-9071-b315d0cda75c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:56.261403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 971/1903] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:56.266368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.308327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] The token cache is being invalidate because project 7c9437720792445e9acc0b877b4d7890 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:32:56.308649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:56.309517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7941cbad-b745-4b1f-9071-b315d0cda75c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:32:56.311182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 933/1904] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:32:55 2022] DELETE /identity/v3/projects/7c9437720792445e9acc0b877b4d7890 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:56.315840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414afd50-2ef0-4953-a9a5-91fdf40b47f4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.316137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414afd50-2ef0-4953-a9a5-91fdf40b47f4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.316842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414afd50-2ef0-4953-a9a5-91fdf40b47f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.351384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc39f84-7389-42b7-ae53-0193f1795554 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.359658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-414afd50-2ef0-4953-a9a5-91fdf40b47f4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:56.359833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.360689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.360982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-414afd50-2ef0-4953-a9a5-91fdf40b47f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:56.361180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.361547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.362512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.365156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.369936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 934/1905] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:56.377879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.378273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.378624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.404016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:56.405585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:56.416515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 935/1906] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:56.420309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.431750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b9ee64d-36ef-4594-b871-82f32c0cffd6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.445086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.445916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.446656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.446957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.447826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:32:56.448234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.450961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.474172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 936/1907] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:32:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:32:56.486985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.499284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d6f206-39d1-40fd-b6ec-86638c6c54f2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.505022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.514195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.514641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.515621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.515981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.518509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.526883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 972/1908] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/projects => generated 387 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:56.531450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.533432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:56.624352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82a61b4e-958b-447d-9f8f-f7f61248e632 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.629472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.630067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.630413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.630996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.632048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.634102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.712552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 937/1909] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/projects => generated 377 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:56.717204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.761082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 973/1910] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/users => generated 294 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:32:56.765436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.777267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4c5a94-42db-4a47-b2fe-993919359819 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.805034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54fbdd9a-f56e-437c-ab92-1eaa812f9150 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.811214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.811900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.812182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.812520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.813255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.815208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.848526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.849771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.851422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.851716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.852472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.854354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.865317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 974/1911] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:56.869477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.880495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a59988-192d-4c51-be19-f4de3b168c37 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.886756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.887501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.887789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.888068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.888936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.891397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.901456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 975/1912] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:56.905730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:56.916634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794c7cc1-40f0-41ba-99d4-ee07f9edf55f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:56.921504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50024573-cada-4f04-b642-16ce3d90519c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:56.922436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024573-cada-4f04-b642-16ce3d90519c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:56.922737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024573-cada-4f04-b642-16ce3d90519c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:56.923010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50024573-cada-4f04-b642-16ce3d90519c admin admin] PATH_INFO: `/v3/projects/fae29a9f6073478380efa3fcf7233aa9/users/ec68aeec921643c8bc94a871b442fddd/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:56.939079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50024573-cada-4f04-b642-16ce3d90519c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fae29a9f6073478380efa3fcf7233aa9, user_id=ec68aeec921643c8bc94a871b442fddd, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:56.941784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50024573-cada-4f04-b642-16ce3d90519c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:56.989205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 938/1913] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:56 2022] POST /identity/v3/users => generated 289 bytes in 272 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:56.994411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.007252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed15b0e-5042-4829-a1d7-dab6a1b8f40c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.012392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.013105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.013465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.013745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.014483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.016119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.028274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 939/1914] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:57.035571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.060637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 976/1915] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:56 2022] PUT /identity/v3/projects/fae29a9f6073478380efa3fcf7233aa9/users/ec68aeec921643c8bc94a871b442fddd/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:57.060841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c2f7da9-7860-41f3-90db-cc5afb0923c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.065501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.065794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.066115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.101266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] MFA Rules not processed for user `ec68aeec921643c8bc94a871b442fddd`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:57.102584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:57.161298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.162975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.163748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.164038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.164898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.166689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.178791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 940/1916] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:57.180852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 977/1917] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:57.182813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.185279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.195177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cff1d2e-0619-4ebe-a8bb-897340773ab0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.199481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b87431e-39d4-4291-886a-2fdb6737c25f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.200393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd78382-80c7-4edb-89d2-e257ed9f895c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.201612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.201737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd78382-80c7-4edb-89d2-e257ed9f895c admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.201737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd78382-80c7-4edb-89d2-e257ed9f895c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.201889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd78382-80c7-4edb-89d2-e257ed9f895c admin admin] PATH_INFO: `/v3/projects/a33f8a6d53eb413296cb04fa4e12bc89/users/302f8036e1a947bfbdb29c92d788043b/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.201961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.202257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.202621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.203389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.205464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.218139 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd78382-80c7-4edb-89d2-e257ed9f895c admin admin] RBAC: Authorizing `identity:create_grant(project_id=a33f8a6d53eb413296cb04fa4e12bc89, user_id=302f8036e1a947bfbdb29c92d788043b, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.221082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd78382-80c7-4edb-89d2-e257ed9f895c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.408654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 978/1918] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:32:57 2022] POST /identity/v3/projects => generated 385 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:57.408858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 941/1919] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:57 2022] PUT /identity/v3/projects/a33f8a6d53eb413296cb04fa4e12bc89/users/302f8036e1a947bfbdb29c92d788043b/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:32:57.408963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.412916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.413666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.414126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.443092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] MFA Rules not processed for user `302f8036e1a947bfbdb29c92d788043b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:57.444829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:57.491568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ae1720d-7f14-49bb-9b03-473e1e5c6fe7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.497474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.498164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.498455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.498738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.499445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.501426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.527269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 942/1920] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:32:57.539683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.541161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:57.619808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2597a5b7-31d6-4ab7-b59c-37f91ebac06a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.623581 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.624242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.624528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.624840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.634331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.635992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.644463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 979/1921] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:32:57 2022] POST /identity/v3/users => generated 299 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:32:57.651827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 943/1922] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:32:57.651827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.662921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50f4d73f-648e-48e8-978d-f4f5dfd201d7 service glance] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.668398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.669283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.669745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.670274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.671150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.673120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.686484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 944/1923] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:32:57.691793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.702494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ee690b-049a-4525-bc28-4987456d152c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.711893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.713128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.713413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.714297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] PATH_INFO: `/v3/projects/7c7df478ccad482b92916aa2cceed79d/users/f39dc1d1b62f4298849885fbe753f14b/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.737708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7df478ccad482b92916aa2cceed79d, user_id=f39dc1d1b62f4298849885fbe753f14b, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.741741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.850563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 980/1924] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:57 2022] PUT /identity/v3/projects/7c7df478ccad482b92916aa2cceed79d/users/f39dc1d1b62f4298849885fbe753f14b/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:57.854424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.868006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa055bb-5f96-49da-9fa8-f0b22321d287 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.941258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107b7664-ee03-41d7-98ec-ddca426b29a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.942417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7664-ee03-41d7-98ec-ddca426b29a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.945590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7664-ee03-41d7-98ec-ddca426b29a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.945590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107b7664-ee03-41d7-98ec-ddca426b29a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.945590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b7664-ee03-41d7-98ec-ddca426b29a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.946204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107b7664-ee03-41d7-98ec-ddca426b29a1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.961938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 945/1925] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:32:57 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:32:57.963141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:57.975373 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d215dc1-91f9-4730-9531-0a0488747536 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:57.980295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:57.981354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:57.981706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:57.982109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] PATH_INFO: `/v3/projects/7c7df478ccad482b92916aa2cceed79d/users/f39dc1d1b62f4298849885fbe753f14b/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:57.984195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c7df478ccad482b92916aa2cceed79d, user_id=f39dc1d1b62f4298849885fbe753f14b, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:57.986943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:57.999166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 981/1926] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:32:57 2022] PUT /identity/v3/projects/7c7df478ccad482b92916aa2cceed79d/users/f39dc1d1b62f4298849885fbe753f14b/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:32:58.010844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:58.011513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:58.011513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:58.055273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] MFA Rules not processed for user `f39dc1d1b62f4298849885fbe753f14b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:32:58.056656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:58.128547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 946/1927] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:32:58 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:32:58.141419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:32:58.143170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:32:58.160222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:32:58.206512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd906629-4e4f-4543-a88f-0a799ae6cf4e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:32:58.211338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:32:58.212009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:32:58.212295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:32:58.212592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:32:58.222851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:32:58.224525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:32:58.234787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 982/1928] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:32:58 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:32:59.045900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:00.045130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:00.057815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802c96c8-c1c0-4cc7-9fb6-d1d0ae94d653 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:00.061889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:00.062621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:00.062908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:00.063212 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:00.065426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:00.132839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:00.134293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:00.144277 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 947/1929] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:00 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:00.180234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=2,keystone:DELETE=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:00.774981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:02.100154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:02.102543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:02.105047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:02.206048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52f6b5b-80d9-4753-af87-980f45740770 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:02.211608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:02.212491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:02.212805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:02.213079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] PATH_INFO: `/v3/users/1d30c6c8fd1248908948c1a23ebf15b8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:02.214260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d30c6c8fd1248908948c1a23ebf15b8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:02.216237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:02.222412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:02.437853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d30c6c8fd1248908948c1a23ebf15b8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:02.438522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d30c6c8fd1248908948c1a23ebf15b8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:02.439149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d30c6c8fd1248908948c1a23ebf15b8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:02.454103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d30c6c8fd1248908948c1a23ebf15b8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:02.463065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7beedbeb-fd99-4433-81cc-f43efe30d90c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d30c6c8fd1248908948c1a23ebf15b8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:02.589441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 983/1930] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:02 2022] DELETE /identity/v3/users/1d30c6c8fd1248908948c1a23ebf15b8 => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:02.892337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:02.904489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb6617b-a583-41f8-9be9-5073abee2f65 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:02.966057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:02.966941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:02.967234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:02.967515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] PATH_INFO: `/v3/projects/3f643c268087457e88f3ddabe8715a87` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:02.968802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f643c268087457e88f3ddabe8715a87)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:02.970852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.086696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f643c268087457e88f3ddabe8715a87', 'request_id': 'req-1b4b0778-c44f-49c7-a10b-4812c304cf0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:03.093106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f643c268087457e88f3ddabe8715a87', 'request_id': 'req-1b4b0778-c44f-49c7-a10b-4812c304cf0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:03.099050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f643c268087457e88f3ddabe8715a87', 'request_id': 'req-1b4b0778-c44f-49c7-a10b-4812c304cf0f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:03.170513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] The token cache is being invalidate because project 3f643c268087457e88f3ddabe8715a87 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:03.170974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:03.171720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1b4b0778-c44f-49c7-a10b-4812c304cf0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:03.173594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 948/1931] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:02 2022] DELETE /identity/v3/projects/3f643c268087457e88f3ddabe8715a87 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:03.179653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.180057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.180288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.225928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:03.227470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:03.296326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 984/1932] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:03.305571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.306159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.306159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.324776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:03.325990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:03.334711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 949/1933] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:03.339039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:03.348893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90de189-c1de-41f6-9a8a-d64a260bd99e None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:03.354762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:03.355596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.355899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.356170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.356953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:03.357263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:03.358871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.373046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 985/1934] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:33:03.377239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:03.387027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1008dd5-904e-406b-b274-058c04717c1c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:03.391833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:03.392501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.392817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.393095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.393821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:03.395734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.496131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 950/1935] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:03 2022] POST /identity/v3/projects => generated 363 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:03.497492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:03.511246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5617362d-ab50-4555-b66c-2e75b0ebb43e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:03.579590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:03.580430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.580771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.581152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.581851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:03.586158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.754909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 986/1936] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:03 2022] POST /identity/v3/users => generated 282 bytes in 258 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:03.759233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:03.769554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b6a43e8-f524-433a-9296-66a5572cf180 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:03.779775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:03.780594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.780900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.781191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.781942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:03.783582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.796478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 951/1937] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:03.800908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:03.811750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bd9912-71fb-41c6-8255-2836410a9b10 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:03.816785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c82eb25-f89d-4ecd-b895-1793465fcd31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:03.817556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82eb25-f89d-4ecd-b895-1793465fcd31 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.817850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82eb25-f89d-4ecd-b895-1793465fcd31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.818259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c82eb25-f89d-4ecd-b895-1793465fcd31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.819588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82eb25-f89d-4ecd-b895-1793465fcd31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:03.821327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c82eb25-f89d-4ecd-b895-1793465fcd31 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.832311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 987/1938] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:03.837909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:03.848318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba206501-6c97-4fb4-9d55-a90d84cebb52 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:03.854311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:03.855356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.855644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.855923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] PATH_INFO: `/v3/projects/47252f622eb6428386ddc67dd84236d3/users/61542472937c4421a5335e6a95ed823a/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:03.872034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:03.872941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47252f622eb6428386ddc67dd84236d3, user_id=61542472937c4421a5335e6a95ed823a, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:03.875769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:03.954400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:03.962765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 952/1939] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:03 2022] PUT /identity/v3/projects/47252f622eb6428386ddc67dd84236d3/users/61542472937c4421a5335e6a95ed823a/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:03.973344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:03.973637 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:03.973916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.007547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] MFA Rules not processed for user `61542472937c4421a5335e6a95ed823a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:04.008756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:04.077976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 988/1940] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:03 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:04.082830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:04.152222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fb9a73-09fe-480a-9b1d-b7ae7ab60122 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:04.157274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:04.158004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.158467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.158751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.159465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:04.161768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:04.321740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 953/1941] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:04 2022] POST /identity/v3/projects => generated 363 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:04.326048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:04.338295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84334e49-9144-4525-b5f6-748fce1ef941 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:04.404878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:04.405571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.405858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.406141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.406846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:04.408814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:04.549079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 989/1942] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:04 2022] POST /identity/v3/users => generated 288 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:04.560337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:04.571920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537ff072-2ed4-46fa-8d47-5693fabe117d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:04.576856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:04.577580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.577864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.578159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.578913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:04.581446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:04.594633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 954/1943] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:04.601666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:04.612270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82faad7d-8ca9-4e47-ac1e-807f97c0c3ff admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:04.619282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:04.620610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.620903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.621175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] PATH_INFO: `/v3/projects/1ebc8f6f2f7a407a86e7673825670cf4/users/5d1b2490edf34b1f93ac82c02ef8f8c7/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.642013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebc8f6f2f7a407a86e7673825670cf4, user_id=5d1b2490edf34b1f93ac82c02ef8f8c7, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:04.644726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:04.747718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 990/1944] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:04 2022] PUT /identity/v3/projects/1ebc8f6f2f7a407a86e7673825670cf4/users/5d1b2490edf34b1f93ac82c02ef8f8c7/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:04.751652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:04.768592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b383e04b-852e-45e5-b084-29d5a56fd2ac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:04.838759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa24a1-b893-4485-9a51-c2eacc321eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:04.838759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa24a1-b893-4485-9a51-c2eacc321eab admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.838759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa24a1-b893-4485-9a51-c2eacc321eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.838759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa24a1-b893-4485-9a51-c2eacc321eab admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.839471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa24a1-b893-4485-9a51-c2eacc321eab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:04.839471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa24a1-b893-4485-9a51-c2eacc321eab admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:04.854892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 955/1945] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:04 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:04.856351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:04.868556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19721894-c231-4670-80d0-69278bea0ec3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:04.875585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:04.878596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.878890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.879171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] PATH_INFO: `/v3/projects/1ebc8f6f2f7a407a86e7673825670cf4/users/5d1b2490edf34b1f93ac82c02ef8f8c7/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.882280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1ebc8f6f2f7a407a86e7673825670cf4, user_id=5d1b2490edf34b1f93ac82c02ef8f8c7, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:04.885364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:04.897492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 991/1946] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:04 2022] PUT /identity/v3/projects/1ebc8f6f2f7a407a86e7673825670cf4/users/5d1b2490edf34b1f93ac82c02ef8f8c7/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:04.909858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:04.910148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:04.910431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:04.941471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] MFA Rules not processed for user `5d1b2490edf34b1f93ac82c02ef8f8c7`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:04.942775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:04.983298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:05.020226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 956/1947] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:04 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:05.032704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:05.034402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:05.094567 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d44e1324-79b1-4ba5-955f-9955999ebc62 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:05.098474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1e3c7f-9dc5-4d33-a2e4-12df299b2a4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:05.099069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e3c7f-9dc5-4d33-a2e4-12df299b2a4a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:05.100052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e3c7f-9dc5-4d33-a2e4-12df299b2a4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:05.100720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1e3c7f-9dc5-4d33-a2e4-12df299b2a4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:05.110197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e3c7f-9dc5-4d33-a2e4-12df299b2a4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:05.111667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1e3c7f-9dc5-4d33-a2e4-12df299b2a4a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:05.122825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 992/1948] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:05 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:06.131851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:06.380002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:06.451408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caac00cd-e610-4ebd-ba22-d9304ff5e054 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:06.456198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:06.457238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:06.457523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:06.457803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] PATH_INFO: `/v3/users/61542472937c4421a5335e6a95ed823a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:06.459053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=61542472937c4421a5335e6a95ed823a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:06.461223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:06.754407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61542472937c4421a5335e6a95ed823a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:06.762481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61542472937c4421a5335e6a95ed823a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:06.775855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61542472937c4421a5335e6a95ed823a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:06.776636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61542472937c4421a5335e6a95ed823a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:06.897621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61542472937c4421a5335e6a95ed823a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:06.905807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 957/1949] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:06 2022] DELETE /identity/v3/users/61542472937c4421a5335e6a95ed823a => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:06.908309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:06.910971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:06.911261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:06.911544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:06.930727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:06.931956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:06.995301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 993/1950] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:07.006331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:07.007867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:07.105959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644ffbe7-f0f3-4f28-a048-c60d9e9fe7ef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:07.111915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:07.113632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:07.113889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:07.114176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:07.125843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:07.127504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:07.141521 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 958/1951] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:07.821423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:07.823129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:07.854808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449f3f2b-204e-4b5b-9cf5-b1f010c11175 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:07.862355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:07.863701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:07.864248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:07.864860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] PATH_INFO: `/v3/projects/47252f622eb6428386ddc67dd84236d3` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:07.865728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:07.867123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47252f622eb6428386ddc67dd84236d3)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:07.867631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:07.871650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:07.925837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:07.937953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c24dca-710a-48ca-8a59-7c7d1455fd33 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:07.942273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:07.943554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:07.944077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:07.944418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:07.946466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:08.011698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:08.013191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:08.022974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 959/1952] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:08.038241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:08.046437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47252f622eb6428386ddc67dd84236d3', 'request_id': 'req-2aac2b64-e0de-418a-b787-575451354483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.055299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47252f622eb6428386ddc67dd84236d3', 'request_id': 'req-2aac2b64-e0de-418a-b787-575451354483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.061407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47252f622eb6428386ddc67dd84236d3', 'request_id': 'req-2aac2b64-e0de-418a-b787-575451354483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.148916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] The token cache is being invalidate because project 47252f622eb6428386ddc67dd84236d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:08.149390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.150288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.151965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 994/1953] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:07 2022] DELETE /identity/v3/projects/47252f622eb6428386ddc67dd84236d3 => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:08.156167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:08.167904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0575e5-5b63-400e-8991-5d60723d37b9 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:08.247171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:08.247777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:08.248979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:08.250637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] PATH_INFO: `/v3/users/5d1b2490edf34b1f93ac82c02ef8f8c7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:08.252146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d1b2490edf34b1f93ac82c02ef8f8c7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:08.254259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:08.474917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d1b2490edf34b1f93ac82c02ef8f8c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.483362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d1b2490edf34b1f93ac82c02ef8f8c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.499237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d1b2490edf34b1f93ac82c02ef8f8c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.504667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d1b2490edf34b1f93ac82c02ef8f8c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.622437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9a241d89-c6d5-410f-9496-77ab171fbe7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d1b2490edf34b1f93ac82c02ef8f8c7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:08.629173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 960/1954] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:08 2022] DELETE /identity/v3/users/5d1b2490edf34b1f93ac82c02ef8f8c7 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:09.406466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:09.408401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:09.422654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aac2b64-e0de-418a-b787-575451354483 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:09.489647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:09.516662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:09.517948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:09.518351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:09.518614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] PATH_INFO: `/v3/projects/1ebc8f6f2f7a407a86e7673825670cf4` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:09.520185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1ebc8f6f2f7a407a86e7673825670cf4)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:09.525428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:09.543206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:09.694513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1ebc8f6f2f7a407a86e7673825670cf4', 'request_id': 'req-7ac1d0cf-045d-4862-932f-a307e9cc77e5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:09.701870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1ebc8f6f2f7a407a86e7673825670cf4', 'request_id': 'req-7ac1d0cf-045d-4862-932f-a307e9cc77e5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:09.708319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1ebc8f6f2f7a407a86e7673825670cf4', 'request_id': 'req-7ac1d0cf-045d-4862-932f-a307e9cc77e5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:09.794549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] The token cache is being invalidate because project 1ebc8f6f2f7a407a86e7673825670cf4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:09.794853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:09.795556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7ac1d0cf-045d-4862-932f-a307e9cc77e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:09.797204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 995/1955] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:09 2022] DELETE /identity/v3/projects/1ebc8f6f2f7a407a86e7673825670cf4 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:09.802221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:09.803176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:09.803598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:09.823312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:09.829063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:09.926311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 961/1956] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:09.936842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:09.937133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:09.937418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:09.956721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:09.957965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:09.972776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 996/1957] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:09.975131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:09.992968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec56bb1-d9e2-4254-9616-af8a2f764de6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:09.998020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:09.998740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:09.999032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:09.999382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.000224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:10.000516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.002259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:10.018480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 962/1958] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:33:10.023595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:10.033327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b8ffbe-3a12-4469-927f-44ea0bb3cd88 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:10.039170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:10.039856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.040149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.040433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.041200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.043161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:10.172086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 997/1959] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:10 2022] POST /identity/v3/projects => generated 365 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:10.176835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:10.188889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a76c22-4fbd-4704-95df-f4992a6f20bf admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:10.256271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:10.257093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.257378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.257658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.258398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.260374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:10.360387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 963/1960] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:10 2022] POST /identity/v3/users => generated 283 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:10.364598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:10.376161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-984cb8c5-274e-4463-8ee2-1591c6da9d1b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:10.391478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:10.392283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.392463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.392720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.393510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.395162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:10.409556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 998/1961] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:10.410815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:10.423203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eae56ad-9a2a-4672-a55d-25b70e48dc35 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:10.428618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790c9ca1-1064-43f4-a8f8-a6dfa272517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:10.429237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c9ca1-1064-43f4-a8f8-a6dfa272517f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.429522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c9ca1-1064-43f4-a8f8-a6dfa272517f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.429802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790c9ca1-1064-43f4-a8f8-a6dfa272517f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.430540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c9ca1-1064-43f4-a8f8-a6dfa272517f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.432429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790c9ca1-1064-43f4-a8f8-a6dfa272517f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:10.444443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 964/1962] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:10.449596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:10.462984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366384a3-8a2b-4795-a418-8334ea3ac6ee admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:10.469796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:10.471204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.471497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.471771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] PATH_INFO: `/v3/projects/3981e2e44f974b46bf123f8a6553e563/users/d9055cbd300d4dceaf61c06fc2ab4c6c/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.501163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3981e2e44f974b46bf123f8a6553e563, user_id=d9055cbd300d4dceaf61c06fc2ab4c6c, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.510001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:10.641909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 999/1963] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:10 2022] PUT /identity/v3/projects/3981e2e44f974b46bf123f8a6553e563/users/d9055cbd300d4dceaf61c06fc2ab4c6c/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:10.653779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.654264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.654353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.684882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] MFA Rules not processed for user `d9055cbd300d4dceaf61c06fc2ab4c6c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:10.686041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:10.760327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 965/1964] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:10.764894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:10.821397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=5,keystone:DELETE=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:10.840093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c67520e-3e9d-472e-904b-569b96cffbb2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:10.846030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:10.847027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:10.848829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:10.850041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:10.851069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:10.853056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:11.172441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1000/1965] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:10 2022] POST /identity/v3/projects => generated 367 bytes in 408 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:11.176600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:11.187832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a5c9c8-141f-4414-bbcc-6bb07cc0af21 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:11.254302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:11.255088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:11.255751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:11.255751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:11.257058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:11.259031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:11.423271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 966/1966] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:11 2022] POST /identity/v3/users => generated 290 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:11.427436 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:11.438896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99fe6568-aa31-465f-9eea-21843edc29c0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:11.446780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:11.447503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:11.447688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:11.447967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:11.448754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:11.450411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:11.463043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1001/1967] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:11.468893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:11.486937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c8f055-6205-48bd-bbf9-69cfb0159506 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:11.491941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:11.492896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:11.493243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:11.493722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] PATH_INFO: `/v3/projects/c3db190470144975bcc5542900abbf9c/users/b56d2b6025f649899d794f609d5fe463/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:11.511337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3db190470144975bcc5542900abbf9c, user_id=b56d2b6025f649899d794f609d5fe463, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:11.514087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:11.657516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 967/1968] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:11 2022] PUT /identity/v3/projects/c3db190470144975bcc5542900abbf9c/users/b56d2b6025f649899d794f609d5fe463/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:11.661725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:11.672549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897e7ffa-a77a-404e-939d-3bb0e7c2af1e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:11.732075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206be95-b06f-4997-a293-f327fcc5ff58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:11.732735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206be95-b06f-4997-a293-f327fcc5ff58 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:11.732982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206be95-b06f-4997-a293-f327fcc5ff58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:11.733261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206be95-b06f-4997-a293-f327fcc5ff58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:11.733998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206be95-b06f-4997-a293-f327fcc5ff58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:11.735628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f206be95-b06f-4997-a293-f327fcc5ff58 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:11.747161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1002/1969] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:11.751658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:11.762971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22be4f43-da01-42dd-ae25-15d43e9d5360 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:11.767944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:11.768862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:11.769179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:11.769427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] PATH_INFO: `/v3/projects/c3db190470144975bcc5542900abbf9c/users/b56d2b6025f649899d794f609d5fe463/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:11.771882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3db190470144975bcc5542900abbf9c, user_id=b56d2b6025f649899d794f609d5fe463, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:11.774612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:11.785948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 968/1970] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:11 2022] PUT /identity/v3/projects/c3db190470144975bcc5542900abbf9c/users/b56d2b6025f649899d794f609d5fe463/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:11.797536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:11.803435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:11.803435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:11.829404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] MFA Rules not processed for user `b56d2b6025f649899d794f609d5fe463`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:11.832014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:11.915497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1003/1971] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:11 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:11.927936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:11.929934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:11.996235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e60e796f-32c3-4636-b58c-493805507814 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:12.000857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:12.003457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:12.003457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:12.003457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:12.003785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:12.018631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:12.020132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:12.031358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 969/1972] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:11 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:12.246807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:12.258893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edeeb0-c219-40ea-9f39-373fda390dfa None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:12.262712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b319cce-69d2-4fa1-a70c-a847f8859c8e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:12.263371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319cce-69d2-4fa1-a70c-a847f8859c8e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:12.263689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319cce-69d2-4fa1-a70c-a847f8859c8e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:12.264001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b319cce-69d2-4fa1-a70c-a847f8859c8e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:12.326257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b319cce-69d2-4fa1-a70c-a847f8859c8e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:12.327743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b319cce-69d2-4fa1-a70c-a847f8859c8e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:12.337607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1004/1973] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:12 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:13.415447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=12,keystone:SELECT=85,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:13.822707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:14.209134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:14.845075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:14.846551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:14.913467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d53dc4-1ed1-4ccf-8404-d00587fc408c service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:14.918252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:14.919136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:14.919415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:14.919691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] PATH_INFO: `/v3/users/912f6c1468884b25a5fd4ddb2677378c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:14.921066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=912f6c1468884b25a5fd4ddb2677378c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:14.926993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:15.159839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '912f6c1468884b25a5fd4ddb2677378c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:15.167482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '912f6c1468884b25a5fd4ddb2677378c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:15.181398 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '912f6c1468884b25a5fd4ddb2677378c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:15.182150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '912f6c1468884b25a5fd4ddb2677378c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:15.222805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:15.257750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '912f6c1468884b25a5fd4ddb2677378c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:15.264424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 970/1974] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:14 2022] DELETE /identity/v3/users/912f6c1468884b25a5fd4ddb2677378c => generated 0 bytes in 420 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:15.269120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:15.269418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:15.269695 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:15.289235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:15.292715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:15.340314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:15.362665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1005/1975] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:15.373690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:15.375226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:15.437986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbbddde-a0d9-449c-9c98-1df65d7e07a4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:15.440710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:15.441368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:15.441651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:15.441928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:15.452035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:15.453530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:15.463030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 971/1976] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:15.931746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:15.931746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:15.946606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a5fa1e-d0e7-4bcb-83ef-4a5772942009 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:15.951602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:15.953574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:15.954214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:15.954505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] PATH_INFO: `/v3/projects/8ebd38a39c064f91b80fd795ee7f8801` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:15.955909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ebd38a39c064f91b80fd795ee7f8801)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:15.957956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:16.106235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ebd38a39c064f91b80fd795ee7f8801', 'request_id': 'req-cafbc6fa-ad7a-4a7e-a304-fb070d115530'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.115502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ebd38a39c064f91b80fd795ee7f8801', 'request_id': 'req-cafbc6fa-ad7a-4a7e-a304-fb070d115530'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.123585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ebd38a39c064f91b80fd795ee7f8801', 'request_id': 'req-cafbc6fa-ad7a-4a7e-a304-fb070d115530'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.206523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] The token cache is being invalidate because project 8ebd38a39c064f91b80fd795ee7f8801 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:16.206816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.209484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.209596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1006/1977] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:15 2022] DELETE /identity/v3/projects/8ebd38a39c064f91b80fd795ee7f8801 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:16.213950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:16.224744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdc07c8-97f0-4195-ac90-fd2b008e2139 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:16.290093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:16.290989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:16.290989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:16.290989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] PATH_INFO: `/v3/users/ec68aeec921643c8bc94a871b442fddd` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:16.292905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec68aeec921643c8bc94a871b442fddd)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:16.294909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:16.573944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec68aeec921643c8bc94a871b442fddd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.581014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec68aeec921643c8bc94a871b442fddd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.600359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec68aeec921643c8bc94a871b442fddd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.602845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec68aeec921643c8bc94a871b442fddd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.657389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:16.696730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec68aeec921643c8bc94a871b442fddd'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:16.704302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 972/1978] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:16 2022] DELETE /identity/v3/users/ec68aeec921643c8bc94a871b442fddd => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:17.148396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:17.309711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:17.315682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:17.328926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafbc6fa-ad7a-4a7e-a304-fb070d115530 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:17.335060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:17.400153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:17.401099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:17.401407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:17.401654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] PATH_INFO: `/v3/projects/fae29a9f6073478380efa3fcf7233aa9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:17.403079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fae29a9f6073478380efa3fcf7233aa9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:17.405088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:17.534514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fae29a9f6073478380efa3fcf7233aa9', 'request_id': 'req-e2c8740f-1a38-402e-b827-74f13bbf9c76'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:17.543195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fae29a9f6073478380efa3fcf7233aa9', 'request_id': 'req-e2c8740f-1a38-402e-b827-74f13bbf9c76'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:17.551810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fae29a9f6073478380efa3fcf7233aa9', 'request_id': 'req-e2c8740f-1a38-402e-b827-74f13bbf9c76'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:17.662554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] The token cache is being invalidate because project fae29a9f6073478380efa3fcf7233aa9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:17.662851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:17.663538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:17.665158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1007/1979] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:17 2022] DELETE /identity/v3/projects/fae29a9f6073478380efa3fcf7233aa9 => generated 0 bytes in 357 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:17.669358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:17.670877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:17.685187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741577d9-2f95-4316-8726-ecced759d37f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:17.753931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:17.754832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:17.755118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:17.755987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] PATH_INFO: `/v3/users/f39dc1d1b62f4298849885fbe753f14b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:17.757472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f39dc1d1b62f4298849885fbe753f14b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:17.759445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:17.895114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:18.030539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f39dc1d1b62f4298849885fbe753f14b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.041141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f39dc1d1b62f4298849885fbe753f14b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.060184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f39dc1d1b62f4298849885fbe753f14b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.061355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f39dc1d1b62f4298849885fbe753f14b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.154641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0380c28-04d8-4298-8c04-1932245579e7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f39dc1d1b62f4298849885fbe753f14b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.161734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 973/1980] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:17 2022] DELETE /identity/v3/users/f39dc1d1b62f4298849885fbe753f14b => generated 0 bytes in 493 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:18.700760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:18.706883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:18.707161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:18.723827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c8740f-1a38-402e-b827-74f13bbf9c76 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:18.804386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:18.807322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:18.807322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:18.807322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] PATH_INFO: `/v3/projects/7c7df478ccad482b92916aa2cceed79d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:18.807322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c7df478ccad482b92916aa2cceed79d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:18.809240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:18.910509 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:18.919068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c7df478ccad482b92916aa2cceed79d', 'request_id': 'req-de3e216b-be71-4dce-b60b-37b04f52fba3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.926578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c7df478ccad482b92916aa2cceed79d', 'request_id': 'req-de3e216b-be71-4dce-b60b-37b04f52fba3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.936783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c7df478ccad482b92916aa2cceed79d', 'request_id': 'req-de3e216b-be71-4dce-b60b-37b04f52fba3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:18.969371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:19.044448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] The token cache is being invalidate because project 7c7df478ccad482b92916aa2cceed79d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:19.044859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:19.045527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-de3e216b-be71-4dce-b60b-37b04f52fba3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:19.047676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1008/1981] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:18 2022] DELETE /identity/v3/projects/7c7df478ccad482b92916aa2cceed79d => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:19.058907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.059245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.059542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.091065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:19.092305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:19.171928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 974/1982] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:19.184940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.186064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.186344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.215146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:19.216416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:19.226938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1009/1983] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:19.231379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:19.247478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b3ca04-7a37-4149-b9f5-c1f050708fa3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:19.253993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:19.254688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.254975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.255303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.256133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:19.256441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:19.258140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:19.273686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 975/1984] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:33:19.278851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:19.292167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfc0270-927d-4edb-9e92-0cbbed5ebae2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:19.297349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:19.300538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.301200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.303316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.304503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:19.306979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:19.326102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:19.450140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1010/1985] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:19 2022] POST /identity/v3/projects => generated 367 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:19.454698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:19.466997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9222ea3-6653-48a3-bde7-923108410f57 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:19.539868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:19.540586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.540894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.541186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.541909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:19.543852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:19.580060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:19.653111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 976/1986] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:19 2022] POST /identity/v3/users => generated 284 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:19.663993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:19.667847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce8d5da-faba-4058-a74d-b7661d56f099 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:19.672510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:19.673246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.673531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.673807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.674540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:19.676203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:19.688248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1011/1987] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:19.696144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:19.706640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5f4f3a-7239-4a7c-a5b3-ca47e3540928 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:19.712910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2db72d0-7c72-46ce-a605-5ead04fba8a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:19.713591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db72d0-7c72-46ce-a605-5ead04fba8a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.713873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db72d0-7c72-46ce-a605-5ead04fba8a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.714149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2db72d0-7c72-46ce-a605-5ead04fba8a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.714890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db72d0-7c72-46ce-a605-5ead04fba8a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:19.716933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2db72d0-7c72-46ce-a605-5ead04fba8a7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:19.729329 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 977/1988] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:19 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:19.736630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:19.747640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e146870f-61d5-4ebe-a41e-677fa42ca087 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:19.753066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:19.753971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.754259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.754542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] PATH_INFO: `/v3/projects/c6c6ba32d8a84b9991fc9f0a7f14e463/users/ae7033cebdb0481ebf34126c19581cfa/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.772859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6c6ba32d8a84b9991fc9f0a7f14e463, user_id=ae7033cebdb0481ebf34126c19581cfa, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:19.774915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:19.883805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1012/1989] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:19 2022] PUT /identity/v3/projects/c6c6ba32d8a84b9991fc9f0a7f14e463/users/ae7033cebdb0481ebf34126c19581cfa/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:19.895026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:19.895319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:19.895635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:19.926046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] MFA Rules not processed for user `ae7033cebdb0481ebf34126c19581cfa`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:19.927375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:19.995723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 978/1990] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:19 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:20.009025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:20.010754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:20.069371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499bd0-054b-4641-843b-512c628f429c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:20.073028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4ed5be-ea97-4246-b1aa-ec1eb134ba03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:20.073694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed5be-ea97-4246-b1aa-ec1eb134ba03 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:20.073978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed5be-ea97-4246-b1aa-ec1eb134ba03 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:20.074254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4ed5be-ea97-4246-b1aa-ec1eb134ba03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:20.084549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed5be-ea97-4246-b1aa-ec1eb134ba03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:20.087041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4ed5be-ea97-4246-b1aa-ec1eb134ba03 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:20.096794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1013/1991] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:20 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:27.605075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:27.606443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:27.695648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba6e428-a6b4-4105-93de-aee04a010728 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:27.703994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:27.705056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:27.705419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:27.705770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] PATH_INFO: `/v3/users/d9055cbd300d4dceaf61c06fc2ab4c6c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:27.707447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9055cbd300d4dceaf61c06fc2ab4c6c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:27.710809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:27.740868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:27.924592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9055cbd300d4dceaf61c06fc2ab4c6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:27.934297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9055cbd300d4dceaf61c06fc2ab4c6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:27.947730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9055cbd300d4dceaf61c06fc2ab4c6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:27.948560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9055cbd300d4dceaf61c06fc2ab4c6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.023494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9055cbd300d4dceaf61c06fc2ab4c6c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.035395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 979/1992] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:27 2022] DELETE /identity/v3/users/d9055cbd300d4dceaf61c06fc2ab4c6c => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:28.041971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:28.042506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:28.042587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:28.064388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:28.066075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:28.151018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1014/1993] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:28.163591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:28.165279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:28.236199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b38341-452d-40d4-b1a1-b5449aec48e1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:28.241928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:28.242615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:28.242936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:28.243238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:28.254075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:28.255601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:28.265871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 980/1994] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:28.478519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:28.483197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:28.505129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605a020a-7ee3-42cc-ac8c-c38221a95c7b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:28.511312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:28.512302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:28.512618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:28.512931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] PATH_INFO: `/v3/users/b6cd7cf8f80749b9ab449776f169ce2a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:28.514261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] RBAC: Authorizing `identity:delete_user(user_id=b6cd7cf8f80749b9ab449776f169ce2a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:28.516207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:28.518077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:28.721986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:28.723683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:28.741537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c154121-f57b-4b11-9434-9f4c9d77190f service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:28.747973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:28.748888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:28.749170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:28.749441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] PATH_INFO: `/v3/projects/3981e2e44f974b46bf123f8a6553e563` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:28.750676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3981e2e44f974b46bf123f8a6553e563)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:28.752769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:28.778246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b6cd7cf8f80749b9ab449776f169ce2a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.778992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b6cd7cf8f80749b9ab449776f169ce2a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.780077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b6cd7cf8f80749b9ab449776f169ce2a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.794068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b6cd7cf8f80749b9ab449776f169ce2a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.801047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3db26fdc-d43c-4a0e-b20b-393c776c867d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b6cd7cf8f80749b9ab449776f169ce2a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.907920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1015/1995] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:28 2022] DELETE /identity/v3/users/b6cd7cf8f80749b9ab449776f169ce2a => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:28.913766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:28.914062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:28.914349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:28.919626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3981e2e44f974b46bf123f8a6553e563', 'request_id': 'req-475a15d6-1f41-4728-bb1e-a19154a3cab8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.926235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3981e2e44f974b46bf123f8a6553e563', 'request_id': 'req-475a15d6-1f41-4728-bb1e-a19154a3cab8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.932345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3981e2e44f974b46bf123f8a6553e563', 'request_id': 'req-475a15d6-1f41-4728-bb1e-a19154a3cab8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:28.933235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:28.934465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:29.002541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1016/1996] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:29.013087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:29.014570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:29.034944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] The token cache is being invalidate because project 3981e2e44f974b46bf123f8a6553e563 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:29.035251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.035937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.041529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 981/1997] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:28 2022] DELETE /identity/v3/projects/3981e2e44f974b46bf123f8a6553e563 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:29.042587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:29.044033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:29.081619 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:29.090041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cefde1-ea2f-489d-bd93-80edf583f20d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:29.093779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:29.095148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:29.097030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:29.097030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:29.101754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:29.121601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:29.124134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:29.131788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a15d6-1f41-4728-bb1e-a19154a3cab8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:29.137373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:29.138280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:29.138593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:29.138773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1017/1998] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:29.138901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] PATH_INFO: `/v3/users/b56d2b6025f649899d794f609d5fe463` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:29.140120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b56d2b6025f649899d794f609d5fe463)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:29.142461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:29.374151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b56d2b6025f649899d794f609d5fe463'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.385659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b56d2b6025f649899d794f609d5fe463'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.400525 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b56d2b6025f649899d794f609d5fe463'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.401459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b56d2b6025f649899d794f609d5fe463'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.524546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2b305db2-62be-4f3b-adb3-05ddd347f865 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b56d2b6025f649899d794f609d5fe463'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.531651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 982/1999] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:29 2022] DELETE /identity/v3/users/b56d2b6025f649899d794f609d5fe463 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:29.618214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:29.620439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:29.640714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3ca845-a625-4521-9e40-a8d60664d336 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:29.668747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:29.714841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:29.715710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:29.716068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:29.716405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] PATH_INFO: `/v3/projects/9b1bce8c77b6416d972e8e935d692663` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:29.717920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b1bce8c77b6416d972e8e935d692663)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:29.719924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:29.843491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b1bce8c77b6416d972e8e935d692663', 'request_id': 'req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.851289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b1bce8c77b6416d972e8e935d692663', 'request_id': 'req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.857993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b1bce8c77b6416d972e8e935d692663', 'request_id': 'req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.986530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] The token cache is being invalidate because project 9b1bce8c77b6416d972e8e935d692663 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:29.986825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.987471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7f8d5de0-4d7d-4a26-bb35-8585e21a7ad8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:29.989077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1018/2000] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:29 2022] DELETE /identity/v3/projects/9b1bce8c77b6416d972e8e935d692663 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:29.995073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:29.995525 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:29.995627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.016078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:30.017341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.084255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 983/2001] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:30.092379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.092859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.093096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.098242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.100195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.117320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-686cf1e7-1956-4cac-863e-91bbac62e131 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.118382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:30.119872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.124032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.125595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.125883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.126168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] PATH_INFO: `/v3/projects/c3db190470144975bcc5542900abbf9c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.127866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3db190470144975bcc5542900abbf9c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.129493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.132152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1019/2002] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:30.133290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.142564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85d8861-03af-4318-b084-90d7056d79a5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.147507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.148503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.149035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.149908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.150726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:30.151262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.152910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.166412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1020/2003] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:33:30.170574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.182163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3af0464-e43f-475a-9057-3d3b7b81ea97 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.192164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.193505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.194048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.194706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.196272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.199193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.270614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:30.281467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3db190470144975bcc5542900abbf9c', 'request_id': 'req-6770ce62-0123-482f-a942-df438c33ccde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:30.289274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3db190470144975bcc5542900abbf9c', 'request_id': 'req-6770ce62-0123-482f-a942-df438c33ccde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:30.295797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3db190470144975bcc5542900abbf9c', 'request_id': 'req-6770ce62-0123-482f-a942-df438c33ccde'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:30.344791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1021/2004] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:30.349602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.372130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] The token cache is being invalidate because project c3db190470144975bcc5542900abbf9c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:30.372437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:30.374386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6770ce62-0123-482f-a942-df438c33ccde admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:30.374658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 984/2005] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:30 2022] DELETE /identity/v3/projects/c3db190470144975bcc5542900abbf9c => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:30.380532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ff487-090a-4d16-9205-b99c19fda4b5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.380935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ff487-090a-4d16-9205-b99c19fda4b5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.381197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-279ff487-090a-4d16-9205-b99c19fda4b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.401447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-279ff487-090a-4d16-9205-b99c19fda4b5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:30.402722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-279ff487-090a-4d16-9205-b99c19fda4b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.418737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fa61332-19ff-4409-af8e-92b14c32b45f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.423795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.424534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.424857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.425130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.425903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.428461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.469564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 985/2006] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:30.478199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.478488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.478896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.502731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:30.503999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.512680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 986/2007] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:30.515656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:30.517593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.523079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1022/2008] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/users => generated 287 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:30.527115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.529040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.531353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d2816f-ae7d-4e99-9965-f023204e637f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.537501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.542107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.542107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.542107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.542107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:30.542107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.543765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.543958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f95c0e9-9edd-43c1-9172-4e22ffefef87 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.550054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.551083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.551266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.551648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.554735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.554911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.566921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1023/2009] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:30.571510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.574737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 987/2010] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:33:30.579516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.584970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a9f4b9-f63f-474d-ab82-0d2e4001a4b6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.590413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.591112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.591399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.591679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.592995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.597680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.597837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a87036-fb18-46ea-b87b-6a69b71398ca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.604796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c5d58-586d-492d-88b1-56c37872cbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.605874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c5d58-586d-492d-88b1-56c37872cbbf admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.606371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c5d58-586d-492d-88b1-56c37872cbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.606811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c5d58-586d-492d-88b1-56c37872cbbf admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.607656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c5d58-586d-492d-88b1-56c37872cbbf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.610607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c5d58-586d-492d-88b1-56c37872cbbf admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.619085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1024/2011] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:30 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:30.623831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.638345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a09f12-5b8d-41cf-9537-1feba20c7d29 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.643432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.644376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.644691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.644970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] PATH_INFO: `/v3/projects/aaacca03609d4f75babb8cf19bbbee89/users/e94bff6a486c4a27bcfbc11effeae8c1/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.662131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=aaacca03609d4f75babb8cf19bbbee89, user_id=e94bff6a486c4a27bcfbc11effeae8c1, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.664843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.775413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1025/2012] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:30 2022] PUT /identity/v3/projects/aaacca03609d4f75babb8cf19bbbee89/users/e94bff6a486c4a27bcfbc11effeae8c1/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:30.775638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 988/2013] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/projects => generated 359 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:30.784843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.789963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.790264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.790545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.830524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] MFA Rules not processed for user `e94bff6a486c4a27bcfbc11effeae8c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:30.831770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.887467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf634e9-a2d2-4341-9bb1-90badd89b1ec admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.907041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 989/2014] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:30.920113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:30.921550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:30.943715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:DELETE=4,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:30.956118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.956828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.957118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.957397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:30.959032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:30.961112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:30.985852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28aeca7e-389b-41e2-a841-95ba335b6c82 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:30.989788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:30.990477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:30.990847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:30.991132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.001674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.003199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.014955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 990/2015] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:31.042287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.045005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1026/2016] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:30 2022] POST /identity/v3/users => generated 280 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:31.048873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.059303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5247a823-554e-4553-8e93-23e7362cf6a3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.059943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7467a0-d550-4701-9cb7-cade3dc9a531 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.064106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.064638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.064820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.065099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.065487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.065637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.065975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.066292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.066393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] PATH_INFO: `/v3/users/e94bff6a486c4a27bcfbc11effeae8c1` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.067536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e94bff6a486c4a27bcfbc11effeae8c1)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.068822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.070788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.082245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1027/2017] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:31.083928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.095230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-126b5aa4-70a4-4da8-9c63-f5d325b8e474 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.100688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.101370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.101751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.102060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.102998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.104639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.115686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1028/2018] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:31.120249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.131062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ec7e38-cbd3-44a9-8d53-34cd273811f6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.136305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95dc0550-4258-43ac-aa63-a7da57fd28e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.137236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc0550-4258-43ac-aa63-a7da57fd28e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.137776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc0550-4258-43ac-aa63-a7da57fd28e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.138995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95dc0550-4258-43ac-aa63-a7da57fd28e3 admin admin] PATH_INFO: `/v3/projects/4c79b454c506468a84d7fd3527c33b09/users/4b34b3890e474553bfd82cb4850adf78/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.154813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc0550-4258-43ac-aa63-a7da57fd28e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4c79b454c506468a84d7fd3527c33b09, user_id=4b34b3890e474553bfd82cb4850adf78, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.157574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95dc0550-4258-43ac-aa63-a7da57fd28e3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.309610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1029/2019] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:31 2022] PUT /identity/v3/projects/4c79b454c506468a84d7fd3527c33b09/users/4b34b3890e474553bfd82cb4850adf78/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:31.321185 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e94bff6a486c4a27bcfbc11effeae8c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:31.321872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45dba2-9011-4936-b593-29abdaae4be2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.322169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45dba2-9011-4936-b593-29abdaae4be2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.322638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a45dba2-9011-4936-b593-29abdaae4be2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.329924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e94bff6a486c4a27bcfbc11effeae8c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:31.330453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:31.349776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e94bff6a486c4a27bcfbc11effeae8c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:31.350505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e94bff6a486c4a27bcfbc11effeae8c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:31.352241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0a45dba2-9011-4936-b593-29abdaae4be2 None None] MFA Rules not processed for user `4b34b3890e474553bfd82cb4850adf78`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:31.354663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0a45dba2-9011-4936-b593-29abdaae4be2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:31.422206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e94bff6a486c4a27bcfbc11effeae8c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:31.430014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1030/2020] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:31 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:31.431314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 991/2021] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:31 2022] DELETE /identity/v3/users/e94bff6a486c4a27bcfbc11effeae8c1 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:31.435652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.436323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.437223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.466000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:31.467215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:31.471102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.473314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:31.552741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1031/2022] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:31.554387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a8c4de-9eec-43cb-90a6-dbdba3917836 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.558693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.558883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.559363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.559772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.560000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.560374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:31.562175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:31.628498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f986cf2b-edec-42af-b72b-db0eaaec5452 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.631772 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:31.633033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.633787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.634078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.634356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.649190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.651092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.651961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.653897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-854cdeb1-6836-4b0f-bfbb-a792a6a9541a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:31.671999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1032/2023] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:31.673783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 992/2024] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:31 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:31.689032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:31.696072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:31.705894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0048bc-d125-4603-8810-cc08ec3c019b service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:31.713239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:31.714262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:31.714548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:31.714826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] PATH_INFO: `/v3/users/4b34b3890e474553bfd82cb4850adf78` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:31.716078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4b34b3890e474553bfd82cb4850adf78)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:31.718069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:32.001622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:32.017419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4b34b3890e474553bfd82cb4850adf78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.018219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4b34b3890e474553bfd82cb4850adf78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.018898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4b34b3890e474553bfd82cb4850adf78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.036310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4b34b3890e474553bfd82cb4850adf78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.045315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4b34b3890e474553bfd82cb4850adf78'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.128749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1033/2025] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:31 2022] DELETE /identity/v3/users/4b34b3890e474553bfd82cb4850adf78 => generated 0 bytes in 441 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:32.142068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.142068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.142068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.153096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:32.154572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:32.201716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:32.203540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:32.243865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 993/2026] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:32.253847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:32.255984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:32.290528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3ad918-30c8-43aa-a874-804835ce69e3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:32.295538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:32.296446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.296801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.297040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] PATH_INFO: `/v3/projects/aaacca03609d4f75babb8cf19bbbee89` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.298240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] RBAC: Authorizing `identity:delete_project(project_id=aaacca03609d4f75babb8cf19bbbee89)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:32.300837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:32.323805 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7c3e8e-4948-464c-9419-f7db67852009 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:32.327656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b3b779-2ee0-4352-a3b2-24e7abe25fb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:32.328317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b779-2ee0-4352-a3b2-24e7abe25fb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.328624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b779-2ee0-4352-a3b2-24e7abe25fb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.328903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b3b779-2ee0-4352-a3b2-24e7abe25fb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.339491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3b779-2ee0-4352-a3b2-24e7abe25fb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:32.341113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b3b779-2ee0-4352-a3b2-24e7abe25fb0 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:32.352824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 994/2027] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:32.445178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aaacca03609d4f75babb8cf19bbbee89', 'request_id': 'req-f9c878d7-0357-4516-ae96-c8ce7433608e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.451948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aaacca03609d4f75babb8cf19bbbee89', 'request_id': 'req-f9c878d7-0357-4516-ae96-c8ce7433608e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.458307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aaacca03609d4f75babb8cf19bbbee89', 'request_id': 'req-f9c878d7-0357-4516-ae96-c8ce7433608e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.564855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] The token cache is being invalidate because project aaacca03609d4f75babb8cf19bbbee89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:32.565859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.566220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f9c878d7-0357-4516-ae96-c8ce7433608e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:32.568343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1034/2028] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:32 2022] DELETE /identity/v3/projects/aaacca03609d4f75babb8cf19bbbee89 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:32.573776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.573776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.573776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.599709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:32.601681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:32.671949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 995/2029] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:32.680127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.680422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.680783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.702336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:32.703572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:32.714013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1035/2030] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:32.719534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:32.734545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c734323a-ee9a-4ca4-8021-6011d597af7e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:32.740338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:32.741370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.741732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.742087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.742930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:32.743311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:32.745323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:32.759595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 996/2031] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:33:32.764002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:32.777669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ebf120e-c038-4c96-9fa1-00d4e17818f1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:32.783297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:32.784171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:32.784465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:32.784774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:32.785643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:32.788763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:32.950965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1036/2032] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:32 2022] POST /identity/v3/projects => generated 383 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:32.956463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:32.967606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ee91bf-d132-424f-b695-b23f8b50450e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:32.985363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:32.991241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:32.994102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:33.019536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:33.051224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.052063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.052331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.052655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.053711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.056097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.094035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8458709a-9991-4ab1-b94e-4f8106a25edc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.099124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.100175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.100476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.100789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] PATH_INFO: `/v3/projects/4c79b454c506468a84d7fd3527c33b09` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.102563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4c79b454c506468a84d7fd3527c33b09)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.104554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.195474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 997/2033] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:32 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:33.199407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.210613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ededb11c-8d3a-47c3-b844-b3df62c1d174 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.215249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.216447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.216447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.216729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.217229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.218850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.232546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 998/2034] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:33.233977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.235048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4c79b454c506468a84d7fd3527c33b09', 'request_id': 'req-5b101212-1ef8-4d95-a534-0621ed290ee6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:33.243703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4c79b454c506468a84d7fd3527c33b09', 'request_id': 'req-5b101212-1ef8-4d95-a534-0621ed290ee6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:33.245526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8274a4-0889-422f-8d84-be5e978f6cc0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.250504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4c79b454c506468a84d7fd3527c33b09', 'request_id': 'req-5b101212-1ef8-4d95-a534-0621ed290ee6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:33.305479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.306174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.306468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.306777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.307522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.309192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.320416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 999/2035] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:33.324642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.328834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] The token cache is being invalidate because project 4c79b454c506468a84d7fd3527c33b09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:33.329137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:33.332522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5b101212-1ef8-4d95-a534-0621ed290ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:33.335355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1037/2036] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:32 2022] DELETE /identity/v3/projects/4c79b454c506468a84d7fd3527c33b09 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:33.338751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35d2d919-6953-4096-8d49-273093bca30a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.340287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b39b-02c0-444f-82e3-8fbc8843c35f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.340763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b39b-02c0-444f-82e3-8fbc8843c35f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.341055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f0b39b-02c0-444f-82e3-8fbc8843c35f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.341169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db50cf6d-a533-4573-913a-83473679ae4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.341882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50cf6d-a533-4573-913a-83473679ae4b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.342164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50cf6d-a533-4573-913a-83473679ae4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.342441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db50cf6d-a533-4573-913a-83473679ae4b admin admin] PATH_INFO: `/v3/projects/562a3f075cb64d668a76838fd4f9c8f5/users/659caf32fa614f64bcd780a264674e33/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.358683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50cf6d-a533-4573-913a-83473679ae4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=562a3f075cb64d668a76838fd4f9c8f5, user_id=659caf32fa614f64bcd780a264674e33, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.359135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-33f0b39b-02c0-444f-82e3-8fbc8843c35f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:33.360518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-33f0b39b-02c0-444f-82e3-8fbc8843c35f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:33.361433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db50cf6d-a533-4573-913a-83473679ae4b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.370212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1038/2037] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:33.377939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.388776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.388776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.399629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:33.400961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:33.409024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1039/2038] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:33.413420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.422837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b315c3a-574f-41f8-9fa4-85a26ba2caf2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.427732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.428237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.434540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.434540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.434540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:33.434540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.434540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.444864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1040/2039] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:33:33.449073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.460846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b81345c-3bb5-4a86-9b19-b6182c5385f5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.462536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1000/2040] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:33 2022] PUT /identity/v3/projects/562a3f075cb64d668a76838fd4f9c8f5/users/659caf32fa614f64bcd780a264674e33/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:33.468186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.468478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.468790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.486069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:33.487454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:33.523100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.523778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.524060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.524368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.525218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.527096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.558092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1001/2041] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:33.568431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.569959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:33.582243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:33.617157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:33.624115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1041/2042] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:33 2022] POST /identity/v3/projects => generated 375 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:33.628215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.637135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9df90b-e5a8-4578-9880-399a7e3451cb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.692438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.693597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.694059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.694341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.694431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564e8366-3282-4d8c-ac05-66472d36af25 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.699162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.704198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.704198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.704198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.704198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.704198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.704872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.706319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.718184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1002/2043] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:33.878452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1042/2044] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:33 2022] POST /identity/v3/users => generated 288 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:33:33.883867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.895085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948e8c56-e342-4817-b922-0650de5ad4ca service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.901130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.901647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.901932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.902214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.903302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.904975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.918602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1003/2045] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:33.922630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.934899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03cdda1-9b95-4f1e-ba9e-3a4afd0ae61c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.940260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae6d43a-61b4-4531-ab6c-535db6f19221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.940857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6d43a-61b4-4531-ab6c-535db6f19221 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.941141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6d43a-61b4-4531-ab6c-535db6f19221 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.941417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae6d43a-61b4-4531-ab6c-535db6f19221 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:33.942593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6d43a-61b4-4531-ab6c-535db6f19221 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:33.945048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae6d43a-61b4-4531-ab6c-535db6f19221 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:33.961781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1043/2046] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:33.966508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:33.979984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de142ae2-917d-497b-b5bf-e542d6098a30 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:33.985916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:33.986820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:33.987101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:33.987379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] PATH_INFO: `/v3/projects/cec53ebddd03478cb47022a8d16d8127/users/8f996ab8b38a4d2c9cf3907fb2918bce/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:34.006958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cec53ebddd03478cb47022a8d16d8127, user_id=8f996ab8b38a4d2c9cf3907fb2918bce, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:34.009767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:34.118951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1004/2047] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:33 2022] PUT /identity/v3/projects/cec53ebddd03478cb47022a8d16d8127/users/8f996ab8b38a4d2c9cf3907fb2918bce/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:34.132431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f5e58-ae54-4fca-a339-938da56b9ea0 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:34.132866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f5e58-ae54-4fca-a339-938da56b9ea0 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:34.133108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990f5e58-ae54-4fca-a339-938da56b9ea0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:34.164552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-990f5e58-ae54-4fca-a339-938da56b9ea0 None None] MFA Rules not processed for user `8f996ab8b38a4d2c9cf3907fb2918bce`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:34.165814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-990f5e58-ae54-4fca-a339-938da56b9ea0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:34.258747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1044/2048] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:34 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:34.269438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:34.271876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:34.357437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5c446d-068c-464a-85e3-70dea4aa33b4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:34.361786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:34.362451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:34.362740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:34.363016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:34.373953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:34.375413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:34.386537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1005/2049] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:35.653066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:35.653606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:35.659426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:35.682319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] MFA Rules not processed for user `659caf32fa614f64bcd780a264674e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:35.683511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:35.744274 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:35.749290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1045/2050] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:35.760746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:35.762170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:35.774075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9430ea82-a660-40c6-9cc7-95fad3bb1148 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:35.777803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3c2b82-b6ca-43e3-9b93-647183922a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:35.778453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c2b82-b6ca-43e3-9b93-647183922a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:35.778733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c2b82-b6ca-43e3-9b93-647183922a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:35.779009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3c2b82-b6ca-43e3-9b93-647183922a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:35.788823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c2b82-b6ca-43e3-9b93-647183922a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:35.790396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3c2b82-b6ca-43e3-9b93-647183922a82 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:35.799855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1006/2051] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:37.935398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:37.937357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:38.015454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e30566-f2bc-45d7-bf53-824c31893bad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:38.022104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:38.023035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:38.023323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:38.023602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] PATH_INFO: `/v3/users/302f8036e1a947bfbdb29c92d788043b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:38.024966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] RBAC: Authorizing `identity:delete_user(user_id=302f8036e1a947bfbdb29c92d788043b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:38.027066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:38.338588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '302f8036e1a947bfbdb29c92d788043b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:38.338937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '302f8036e1a947bfbdb29c92d788043b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:38.339647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '302f8036e1a947bfbdb29c92d788043b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:38.355339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '302f8036e1a947bfbdb29c92d788043b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:38.362394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '302f8036e1a947bfbdb29c92d788043b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:38.532818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1046/2052] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:37 2022] DELETE /identity/v3/users/302f8036e1a947bfbdb29c92d788043b => generated 0 bytes in 598 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:38.537988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927a47-296e-480f-b16c-438f59bc8b47 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:38.538542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927a47-296e-480f-b16c-438f59bc8b47 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:38.538677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92927a47-296e-480f-b16c-438f59bc8b47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:38.545149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:38.559189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-92927a47-296e-480f-b16c-438f59bc8b47 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:38.560619 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-92927a47-296e-480f-b16c-438f59bc8b47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:38.625033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1007/2053] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:38.636849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:38.638250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:38.702872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79d916b8-754d-402e-b2c3-11b676f577da admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:38.709458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:38.710561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:38.711049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:38.711498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:38.728015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:38.731190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:38.749853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1047/2054] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:39.041729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4ed05-4d3e-4a8f-abc8-b8121fe37ff0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.042037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4ed05-4d3e-4a8f-abc8-b8121fe37ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.042325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd4ed05-4d3e-4a8f-abc8-b8121fe37ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.063162 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5fd4ed05-4d3e-4a8f-abc8-b8121fe37ff0 None None] MFA Rules not processed for user `659caf32fa614f64bcd780a264674e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:39.064642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5fd4ed05-4d3e-4a8f-abc8-b8121fe37ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:39.125575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1008/2055] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:39.274362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:39.275946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:39.290126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c1d41b-8e51-4324-88cc-8135d86d3a57 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:39.295746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:39.296677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.296987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.298646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] PATH_INFO: `/v3/projects/a33f8a6d53eb413296cb04fa4e12bc89` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.299196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a33f8a6d53eb413296cb04fa4e12bc89)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:39.301442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:39.322271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:39.468416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a33f8a6d53eb413296cb04fa4e12bc89', 'request_id': 'req-7bb38775-f692-4857-aec9-d2968d65aae5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:39.476755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a33f8a6d53eb413296cb04fa4e12bc89', 'request_id': 'req-7bb38775-f692-4857-aec9-d2968d65aae5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:39.485118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a33f8a6d53eb413296cb04fa4e12bc89', 'request_id': 'req-7bb38775-f692-4857-aec9-d2968d65aae5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:39.570356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] The token cache is being invalidate because project a33f8a6d53eb413296cb04fa4e12bc89 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:39.570807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:39.571978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7bb38775-f692-4857-aec9-d2968d65aae5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:39.573971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1048/2056] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:39 2022] DELETE /identity/v3/projects/a33f8a6d53eb413296cb04fa4e12bc89 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:39.580901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.581440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.581440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.604690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:39.606367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:39.680948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1009/2057] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:39.688347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.692621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.692621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.710204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:39.711446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:39.720425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1049/2058] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:39.726098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:39.735960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0b1011-9132-4c88-abbe-901f005e12fc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:39.740981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:39.741661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.741969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.742265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.742982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:39.743296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:39.745312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:39.758428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1010/2059] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 13:33:39.762858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:39.773991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5440c670-188a-4eb4-add4-53abab959a07 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:39.779349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:39.780059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.780353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.780689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.781441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:39.783356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:39.900519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1050/2060] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:39 2022] POST /identity/v3/projects => generated 375 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:39.905014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:39.919055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2730348-b292-4ad1-97e3-4ffcbba79f08 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:39.963243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:39.989136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:39.990221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:39.990388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:39.990672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:39.991387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:39.993389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.077933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1011/2061] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:39 2022] POST /identity/v3/users => generated 288 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:40.082283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.093579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e12da4-db0c-4e1f-886a-5cc8d581ef98 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.098194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.098932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.099217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.099492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.100211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.101971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.113210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1051/2062] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:40.117413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.128130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39770ac-a592-466d-87d9-8619d4e30cae admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.133614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee72a452-b9eb-4ba0-9e16-26f71179b4f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.134216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72a452-b9eb-4ba0-9e16-26f71179b4f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.134498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72a452-b9eb-4ba0-9e16-26f71179b4f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.134775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee72a452-b9eb-4ba0-9e16-26f71179b4f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.135643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72a452-b9eb-4ba0-9e16-26f71179b4f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.137192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee72a452-b9eb-4ba0-9e16-26f71179b4f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.151730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1012/2063] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:40.154680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.165242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f79631-1c68-40e8-ae5e-2402fc65c02c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.169972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.170864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.171152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.171433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] PATH_INFO: `/v3/projects/2a46e27d98024f30bbd083f291252425/users/0c8d5cf5de6b4427bc7e2e526c4ae3ea/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.189279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a46e27d98024f30bbd083f291252425, user_id=0c8d5cf5de6b4427bc7e2e526c4ae3ea, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.191990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.341391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1052/2064] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:40 2022] PUT /identity/v3/projects/2a46e27d98024f30bbd083f291252425/users/0c8d5cf5de6b4427bc7e2e526c4ae3ea/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:40.352905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.353422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.353931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.396818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] MFA Rules not processed for user `0c8d5cf5de6b4427bc7e2e526c4ae3ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:40.398047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:40.491016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1013/2065] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:40 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:33:40.493767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.560751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d44fa-7436-48f4-9fbf-fd91dbe1ba0f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.565442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.566111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.566399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.566678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.567384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.569891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.685405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1053/2066] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:40 2022] POST /identity/v3/projects => generated 375 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:40.689391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.700483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf843f8f-ae07-4817-9920-1e5478332b6c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.765335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.766032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.766320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.766601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.767379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.769436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.927373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1014/2067] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:40 2022] POST /identity/v3/users => generated 288 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:40.929209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.940818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db97c6e-6bc6-4e66-826f-0e3d5c78fb2d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.945402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.946055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.946336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.946799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.947566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.949223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:40.963386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1054/2068] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:40.965938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:40.977670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a527a408-29f2-4ad6-bbb7-333a5eb34267 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:40.983987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acddab24-91fa-4245-b8ec-c6ee94c7a05f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:40.984963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddab24-91fa-4245-b8ec-c6ee94c7a05f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:40.985319 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddab24-91fa-4245-b8ec-c6ee94c7a05f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:40.985698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acddab24-91fa-4245-b8ec-c6ee94c7a05f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:40.986518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddab24-91fa-4245-b8ec-c6ee94c7a05f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:40.988421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acddab24-91fa-4245-b8ec-c6ee94c7a05f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:41.003896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1015/2069] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:40 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:41.005496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:41.017360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6128553-75bf-4156-9105-3b247bcd6e24 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:41.024530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:41.026219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:41.026798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:41.027411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] PATH_INFO: `/v3/projects/a40ff645bd4a40939d780405de9f9df8/users/a70c0e2b5f77406db4ee4660e0d1d0b3/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:41.053553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:41.058873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a40ff645bd4a40939d780405de9f9df8, user_id=a70c0e2b5f77406db4ee4660e0d1d0b3, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:41.062137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:41.219025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1055/2070] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:41 2022] PUT /identity/v3/projects/a40ff645bd4a40939d780405de9f9df8/users/a70c0e2b5f77406db4ee4660e0d1d0b3/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:41.233369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:41.233791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:41.234358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:41.267412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] MFA Rules not processed for user `a70c0e2b5f77406db4ee4660e0d1d0b3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:41.268672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:41.280535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:41.335881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1016/2071] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:41.340413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:41.406200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3c7dfcd-3468-4481-8ccc-150b8d3d3ec9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:41.410955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:41.411868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:41.412150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:41.412480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:41.413203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:41.415114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:41.572489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1056/2072] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:41 2022] POST /identity/v3/projects => generated 375 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:41.576698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:41.589082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382c8f59-55e3-4f04-b170-19d259f8bdea None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:41.657926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:41.658605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:41.658887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:41.659161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:41.659866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:41.662122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:41.863861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1017/2073] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:41 2022] POST /identity/v3/users => generated 294 bytes in 288 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:41.870024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:41.881953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a461f96-1ffd-48e2-87e2-449733133dcc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:41.886690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:41.887370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:41.887658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:41.887942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:41.888718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:41.890389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:41.902542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1057/2074] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:41.919100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:41.931480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a6e226-825b-43f4-a7c7-f45c5af16f78 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:41.936788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:41.937624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:41.937933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:41.938260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] PATH_INFO: `/v3/projects/73e504cb3a164d92969a6a855ff2c5b9/users/0fc2a401ce2a45908531190440ceaa50/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:41.957416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e504cb3a164d92969a6a855ff2c5b9, user_id=0fc2a401ce2a45908531190440ceaa50, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:41.960286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:42.264558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1018/2075] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:41 2022] PUT /identity/v3/projects/73e504cb3a164d92969a6a855ff2c5b9/users/0fc2a401ce2a45908531190440ceaa50/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:33:42.268757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:42.280588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507e833d-99d4-46a6-a89a-722ea0fc6231 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:42.359107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e8d6b6a-7127-4ff6-b728-582c98c85c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:42.359776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6b6a-7127-4ff6-b728-582c98c85c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:42.360066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6b6a-7127-4ff6-b728-582c98c85c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:42.360351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8d6b6a-7127-4ff6-b728-582c98c85c69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:42.361197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d6b6a-7127-4ff6-b728-582c98c85c69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:42.362821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e8d6b6a-7127-4ff6-b728-582c98c85c69 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:42.377577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1058/2076] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:42 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:42.378656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:42.389168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4143e5-6ecc-452d-a86c-9bf3abd750ea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:42.393688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:42.394598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:42.394882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:42.395158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] PATH_INFO: `/v3/projects/73e504cb3a164d92969a6a855ff2c5b9/users/0fc2a401ce2a45908531190440ceaa50/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:42.397215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=73e504cb3a164d92969a6a855ff2c5b9, user_id=0fc2a401ce2a45908531190440ceaa50, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:42.399857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:42.410409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1019/2077] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:42 2022] PUT /identity/v3/projects/73e504cb3a164d92969a6a855ff2c5b9/users/0fc2a401ce2a45908531190440ceaa50/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:42.421462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:42.421992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:42.422103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:42.452079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] MFA Rules not processed for user `0fc2a401ce2a45908531190440ceaa50`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:42.453349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:42.503721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:42.526950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1059/2078] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:42.561452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:42.561452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:42.604181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040aca0d-9c2a-418e-85cb-14dbbaa58dd6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:42.608035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:42.608729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:42.609018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:42.609297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:42.626784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:42.666597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:42.667967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:42.678183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1020/2079] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:46.220822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:46.223386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:46.243853 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514fb6e2-58a0-414d-91d8-c80eca7ba3b3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:46.248529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:46.249550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:46.250003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:46.250631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:46.327851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:46.329273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:46.340628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1060/2080] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:33:48.612556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:48.622282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81facde4-0890-4f40-b805-6a97ce74a8e8 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:48.625993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd297237-6e43-4200-8f59-ed4fc52a2fac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:48.627301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd297237-6e43-4200-8f59-ed4fc52a2fac service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:48.627627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd297237-6e43-4200-8f59-ed4fc52a2fac service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:48.628009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd297237-6e43-4200-8f59-ed4fc52a2fac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:48.638253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd297237-6e43-4200-8f59-ed4fc52a2fac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:48.639759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd297237-6e43-4200-8f59-ed4fc52a2fac service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:48.650356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1021/2081] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:48 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:49.758786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:49.760246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:49.835433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7535412-a4d3-4d7d-a0c5-3a22d868cb2a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:49.840742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:49.841643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:49.841918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:49.842208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] PATH_INFO: `/v3/users/8f996ab8b38a4d2c9cf3907fb2918bce` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:49.843511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f996ab8b38a4d2c9cf3907fb2918bce)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:49.845507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:50.089902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f996ab8b38a4d2c9cf3907fb2918bce'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:50.090866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f996ab8b38a4d2c9cf3907fb2918bce'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:50.091489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f996ab8b38a4d2c9cf3907fb2918bce'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:50.108843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f996ab8b38a4d2c9cf3907fb2918bce'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:50.118212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f996ab8b38a4d2c9cf3907fb2918bce'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:50.191037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1061/2082] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:33:49 2022] DELETE /identity/v3/users/8f996ab8b38a4d2c9cf3907fb2918bce => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:50.195829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:50.196340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:50.196441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:50.215573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:50.216855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:50.317949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:50.319628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:50.320739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1022/2083] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:50.326201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:50.395100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34050c43-9644-4353-9c8d-3abe6a5ac305 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:50.399271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93fa6dc3-abf7-4ba8-9315-773235b5617c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:50.399935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa6dc3-abf7-4ba8-9315-773235b5617c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:50.400217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa6dc3-abf7-4ba8-9315-773235b5617c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:50.400493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93fa6dc3-abf7-4ba8-9315-773235b5617c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:50.412475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa6dc3-abf7-4ba8-9315-773235b5617c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:50.414118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93fa6dc3-abf7-4ba8-9315-773235b5617c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:50.429729 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1062/2084] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:33:50.905278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:50.907579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:50.922510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd0209-7a1c-40fb-a52e-945ed907ecd5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:50.927791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:50.928815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:50.929093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:50.930974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] PATH_INFO: `/v3/projects/cec53ebddd03478cb47022a8d16d8127` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:50.930974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cec53ebddd03478cb47022a8d16d8127)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:50.932540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.066298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cec53ebddd03478cb47022a8d16d8127', 'request_id': 'req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:51.073644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cec53ebddd03478cb47022a8d16d8127', 'request_id': 'req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:51.079614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cec53ebddd03478cb47022a8d16d8127', 'request_id': 'req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:51.203202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] The token cache is being invalidate because project cec53ebddd03478cb47022a8d16d8127 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:33:51.203869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:51.204093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6bfceceb-0193-48ce-9fc5-c1a41b8bcef1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:33:51.205670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1023/2085] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:33:50 2022] DELETE /identity/v3/projects/cec53ebddd03478cb47022a8d16d8127 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:51.210531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.210931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.211152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.229897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:51.231110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:51.293803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1063/2086] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:51.301535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.302633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.303250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.323186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:51.325006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:51.333776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1024/2087] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:51.338119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:51.347809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc7910c-6e58-47c9-8cd2-b1e2e891bf78 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:51.353431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:51.354096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.354384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.354661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.355513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:33:51.355816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:51.357752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.371150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1064/2088] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:33:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:33:51.377598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:51.387544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171dbd0e-c97c-4153-9f20-e5f1f8b60bd0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:51.392530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:51.393240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.393527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.393822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.394501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:51.397224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.501656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:51.511530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1025/2089] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:33:51 2022] POST /identity/v3/projects => generated 399 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:51.512973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:51.523833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b975e2-c59b-4bc4-aed2-230d514d398a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:51.585455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:51.586150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.586429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.586710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.587422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:51.589397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.706404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1065/2090] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:33:51 2022] POST /identity/v3/users => generated 300 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:33:51.712218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:51.727381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eb28d9-a41b-453b-8bbb-43f889a71c75 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:51.729082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:51.729827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.730120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.730403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.731141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:51.732903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.744941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1026/2091] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:33:51.749068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:51.759412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775e78af-424d-43cf-82e7-3ef8924a0e28 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:51.763999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2287b705-840a-46bc-b519-6096ea213512 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:51.764697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b705-840a-46bc-b519-6096ea213512 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.764991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b705-840a-46bc-b519-6096ea213512 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.765263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2287b705-840a-46bc-b519-6096ea213512 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.765989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287b705-840a-46bc-b519-6096ea213512 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:51.767636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2287b705-840a-46bc-b519-6096ea213512 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.778610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1066/2092] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:33:51 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:33:51.791105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:51.801134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a57383-cee3-4377-9af0-50e2d756d7e7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:51.806351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:51.807234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.807517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.807796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] PATH_INFO: `/v3/projects/ff0ee1edd3184df0a427029ccab53738/users/cb3d1da06ba943a3bc70d62b8e8b8f6b/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.824202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff0ee1edd3184df0a427029ccab53738, user_id=cb3d1da06ba943a3bc70d62b8e8b8f6b, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:51.825601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:51.923700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1027/2093] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:33:51 2022] PUT /identity/v3/projects/ff0ee1edd3184df0a427029ccab53738/users/cb3d1da06ba943a3bc70d62b8e8b8f6b/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:33:51.934117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a633cb-be0f-4600-b043-59e459e4a451 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:51.934406 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a633cb-be0f-4600-b043-59e459e4a451 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:51.934684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a633cb-be0f-4600-b043-59e459e4a451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:51.957789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:33:51.966205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b9a633cb-be0f-4600-b043-59e459e4a451 None None] MFA Rules not processed for user `cb3d1da06ba943a3bc70d62b8e8b8f6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:33:51.967429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b9a633cb-be0f-4600-b043-59e459e4a451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:52.034187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1067/2094] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:33:51 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:33:52.046413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:33:52.048194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:33:52.117565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043a1965-8621-4ce9-b762-20ce911fa4d4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:33:52.121666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:33:52.122313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:33:52.122601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:33:52.122909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:33:52.132351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:33:52.133911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:33:52.143268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1028/2095] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:33:52 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:02.491369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=27 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:02.576270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=41,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:03.248044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f61058-56f2-4221-83f9-d98d96a81413 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:03.248356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f61058-56f2-4221-83f9-d98d96a81413 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:03.248689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f61058-56f2-4221-83f9-d98d96a81413 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:03.275056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-94f61058-56f2-4221-83f9-d98d96a81413 None None] MFA Rules not processed for user `af29f227080147a1a157f88842d39483`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:03.275056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-94f61058-56f2-4221-83f9-d98d96a81413 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:03.328750 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:03.330218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:03.340878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1068/2096] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:03 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:03.413399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142b80c8-3271-4059-b5c4-413aef91edff service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:03.421887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:03.422853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:03.423267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:03.423613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] PATH_INFO: `/v3/users/0c8d5cf5de6b4427bc7e2e526c4ae3ea` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:03.425082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0c8d5cf5de6b4427bc7e2e526c4ae3ea)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:03.427443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:03.653349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0c8d5cf5de6b4427bc7e2e526c4ae3ea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:03.662974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0c8d5cf5de6b4427bc7e2e526c4ae3ea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:03.677308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0c8d5cf5de6b4427bc7e2e526c4ae3ea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:03.678052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0c8d5cf5de6b4427bc7e2e526c4ae3ea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:03.782956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0c8d5cf5de6b4427bc7e2e526c4ae3ea'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:03.791063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1029/2097] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:34:03 2022] DELETE /identity/v3/users/0c8d5cf5de6b4427bc7e2e526c4ae3ea => generated 0 bytes in 463 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:34:03.795792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:03.796319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:03.796445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:03.816135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:03.817578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:03.889250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1069/2098] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:03.899759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:03.901493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:03.968530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c591bb-fd73-4cea-9fda-c4afab055c09 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:03.973154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:03.974887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:03.975183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:03.975469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:03.985195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:03.986672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:03.998821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1030/2099] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:04.563371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:04.563371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:04.576105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f57288-a9a5-4749-90e7-e63553a101c8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:04.581327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:04.582218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:04.582505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:04.583461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] PATH_INFO: `/v3/projects/2a46e27d98024f30bbd083f291252425` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:04.584960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a46e27d98024f30bbd083f291252425)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:04.586920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:04.744038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a46e27d98024f30bbd083f291252425', 'request_id': 'req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:04.751278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a46e27d98024f30bbd083f291252425', 'request_id': 'req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:04.758276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a46e27d98024f30bbd083f291252425', 'request_id': 'req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:04.887127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] The token cache is being invalidate because project 2a46e27d98024f30bbd083f291252425 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:34:04.887824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:04.888089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:04.889912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1070/2100] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:34:04 2022] DELETE /identity/v3/projects/2a46e27d98024f30bbd083f291252425 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:34:04.894021 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:04.904453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48353e75-4664-43ef-a00e-88082996e9b6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:04.930463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:04.973795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:04.974675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:04.975024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:04.975339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] PATH_INFO: `/v3/users/a70c0e2b5f77406db4ee4660e0d1d0b3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:04.976930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a70c0e2b5f77406db4ee4660e0d1d0b3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:04.979261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:05.215715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a70c0e2b5f77406db4ee4660e0d1d0b3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:05.226230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a70c0e2b5f77406db4ee4660e0d1d0b3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:05.241543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a70c0e2b5f77406db4ee4660e0d1d0b3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:05.242303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a70c0e2b5f77406db4ee4660e0d1d0b3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:05.317301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a70c0e2b5f77406db4ee4660e0d1d0b3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:05.324412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1031/2101] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:34:04 2022] DELETE /identity/v3/users/a70c0e2b5f77406db4ee4660e0d1d0b3 => generated 0 bytes in 431 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:05.824122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:05.826700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:05.853200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0e909b-5c2d-4c96-a69e-4b3a0fc88921 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:05.884192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:05.920706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:05.921607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:05.921896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:05.922178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] PATH_INFO: `/v3/projects/a40ff645bd4a40939d780405de9f9df8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:05.923390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=a40ff645bd4a40939d780405de9f9df8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:05.925479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:06.050652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a40ff645bd4a40939d780405de9f9df8', 'request_id': 'req-1a2fec1c-e679-4525-ad0d-e28b386768ae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.058200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a40ff645bd4a40939d780405de9f9df8', 'request_id': 'req-1a2fec1c-e679-4525-ad0d-e28b386768ae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.071556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a40ff645bd4a40939d780405de9f9df8', 'request_id': 'req-1a2fec1c-e679-4525-ad0d-e28b386768ae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.205895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] The token cache is being invalidate because project a40ff645bd4a40939d780405de9f9df8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:34:06.206399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.207110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.208903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1071/2102] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:34:05 2022] DELETE /identity/v3/projects/a40ff645bd4a40939d780405de9f9df8 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:34:06.213149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:06.215471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:06.230112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7efaf-6f71-4c6a-9bc3-e46d8e190dc4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:06.292035 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:06.292962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:06.293253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:06.293541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] PATH_INFO: `/v3/users/0fc2a401ce2a45908531190440ceaa50` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:06.294791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fc2a401ce2a45908531190440ceaa50)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:06.296909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:06.556352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:06.560495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fc2a401ce2a45908531190440ceaa50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.568574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fc2a401ce2a45908531190440ceaa50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.581071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fc2a401ce2a45908531190440ceaa50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.583927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fc2a401ce2a45908531190440ceaa50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.679908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5407906e-0ac8-4290-8646-001a0819a829 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fc2a401ce2a45908531190440ceaa50'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:06.686761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1032/2103] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:34:06 2022] DELETE /identity/v3/users/0fc2a401ce2a45908531190440ceaa50 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:07.130771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:07.132351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:07.148491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2fec1c-e679-4525-ad0d-e28b386768ae admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:07.226929 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:07.227966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:07.228197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:07.228475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] PATH_INFO: `/v3/projects/73e504cb3a164d92969a6a855ff2c5b9` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:07.229760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=73e504cb3a164d92969a6a855ff2c5b9)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:07.231888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:07.357429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '73e504cb3a164d92969a6a855ff2c5b9', 'request_id': 'req-ed911174-382a-4daa-adb1-6f8a7d779d4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:07.365012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '73e504cb3a164d92969a6a855ff2c5b9', 'request_id': 'req-ed911174-382a-4daa-adb1-6f8a7d779d4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:07.371269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:07.372200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '73e504cb3a164d92969a6a855ff2c5b9', 'request_id': 'req-ed911174-382a-4daa-adb1-6f8a7d779d4c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:07.513107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] The token cache is being invalidate because project 73e504cb3a164d92969a6a855ff2c5b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:34:07.513430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:07.514217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ed911174-382a-4daa-adb1-6f8a7d779d4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:07.515956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1072/2104] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:34:07 2022] DELETE /identity/v3/projects/73e504cb3a164d92969a6a855ff2c5b9 => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:07.521405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:07.521897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:07.521987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:07.545485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:07.546691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:07.621577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1033/2105] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:07.624532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:07.624917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:07.625174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:07.655665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:07.657102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:07.666464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1073/2106] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:34:07.681625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:07.682507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46461b6-4269-4d8c-81ed-084bec0b7803 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:07.687684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:07.688440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:07.688809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:07.689097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:07.690024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:34:07.690333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:07.692360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:07.717610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1034/2107] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:34:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:34:07.722416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:07.733968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b159749b-76a2-4683-a74c-d2bdd76bb79f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:07.739142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:07.740354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:07.740699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:07.740985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:07.741696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:07.744069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:07.999999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1074/2108] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:34:07 2022] POST /identity/v3/projects => generated 371 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:08.004220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:08.016278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c9370f-056b-427c-b110-19e858d10da7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:08.082544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:08.083781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:08.084108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:08.084341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:08.085095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:08.087187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:08.244420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1035/2109] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:34:08 2022] POST /identity/v3/users => generated 286 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:08.248640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:08.265101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d22ddb-4114-4d72-bacf-e3ae0d292511 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:08.270896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:08.271936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:08.272066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:08.272341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:08.273132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:08.275048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:08.286429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1075/2110] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:08.290435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:08.300834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f74529b-2d3e-494f-8550-96d44e8d6120 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:08.306561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e0f862-d6d7-48da-bb19-e225a8f47a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:08.307236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f862-d6d7-48da-bb19-e225a8f47a69 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:08.307557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f862-d6d7-48da-bb19-e225a8f47a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:08.307836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e0f862-d6d7-48da-bb19-e225a8f47a69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:08.308584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f862-d6d7-48da-bb19-e225a8f47a69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:08.310252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e0f862-d6d7-48da-bb19-e225a8f47a69 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:08.321641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1036/2111] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:08.325951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:08.336457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41330cb3-2731-4d4f-b584-4e6efdf33b3c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:08.341490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:08.342438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:08.342725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:08.343024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] PATH_INFO: `/v3/projects/b3808ebec9df4a4c937ab961f3683dda/users/740cc82ff2c04228b8ba32ecc034944f/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:08.360518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b3808ebec9df4a4c937ab961f3683dda, user_id=740cc82ff2c04228b8ba32ecc034944f, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:08.363323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:08.474157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1076/2112] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:08 2022] PUT /identity/v3/projects/b3808ebec9df4a4c937ab961f3683dda/users/740cc82ff2c04228b8ba32ecc034944f/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:08.479700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:08.493790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:08.493790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:08.501480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:08.503635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:08.533620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:08.577403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1037/2113] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:08.600262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:08.602616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:08.671069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5940f4-767b-40c3-9ec6-058048082ec4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:08.674969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6a96c7-125d-4847-9971-dfedc2b2524b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:08.677093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a96c7-125d-4847-9971-dfedc2b2524b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:08.677093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a96c7-125d-4847-9971-dfedc2b2524b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:08.677093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6a96c7-125d-4847-9971-dfedc2b2524b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:08.688587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a96c7-125d-4847-9971-dfedc2b2524b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:08.690094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6a96c7-125d-4847-9971-dfedc2b2524b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:08.700685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1077/2114] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:09.983643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:09.985426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:10.000408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba7f6c7-5284-4a9d-bcf7-f738c8c8148a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:10.006287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:10.007460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:10.007760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:10.008366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] PATH_INFO: `/v3/users/cb3d1da06ba943a3bc70d62b8e8b8f6b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:10.009742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb3d1da06ba943a3bc70d62b8e8b8f6b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:10.011709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:10.217077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb3d1da06ba943a3bc70d62b8e8b8f6b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:10.224528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb3d1da06ba943a3bc70d62b8e8b8f6b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:10.244811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb3d1da06ba943a3bc70d62b8e8b8f6b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:10.244811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb3d1da06ba943a3bc70d62b8e8b8f6b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:10.360262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3c438b92-e43f-4edc-9132-fa93f6c18449 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb3d1da06ba943a3bc70d62b8e8b8f6b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:10.369306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1038/2115] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:34:09 2022] DELETE /identity/v3/users/cb3d1da06ba943a3bc70d62b8e8b8f6b => generated 0 bytes in 386 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:10.373940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:10.374422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:10.374548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:10.394466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:10.396641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:10.500612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:10.500612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:10.501270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:10.512383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1078/2116] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:10.532926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:10.538288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:10.562499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] MFA Rules not processed for user `740cc82ff2c04228b8ba32ecc034944f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:10.564940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:10.595756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=3,keystone:DELETE=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:10.647381 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64eca927-27fd-4b39-a9eb-cf7f8143c972 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:10.652727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:10.653412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:10.653698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:10.653999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:10.678214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:10.680815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:10.700049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1039/2117] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:10 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:10.700267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1079/2118] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:10.708045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:10.709582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:10.760440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:10.792775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe49b85-d124-4e66-9c9b-245e5c7016b6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:10.796820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c8d1dd-0ddb-47b2-a240-bef9d04f0ae8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:10.797524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d1dd-0ddb-47b2-a240-bef9d04f0ae8 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:10.797812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d1dd-0ddb-47b2-a240-bef9d04f0ae8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:10.798090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8d1dd-0ddb-47b2-a240-bef9d04f0ae8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:10.809073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d1dd-0ddb-47b2-a240-bef9d04f0ae8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:10.810598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c8d1dd-0ddb-47b2-a240-bef9d04f0ae8 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:10.821475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1040/2119] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:11.515467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:11.515467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:11.528253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8866a-fc4a-40e1-afa4-929163ac5426 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:11.537667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:11.540171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:11.540662 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:11.540988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] PATH_INFO: `/v3/projects/ff0ee1edd3184df0a427029ccab53738` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:11.543753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff0ee1edd3184df0a427029ccab53738)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:11.545945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:11.760690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff0ee1edd3184df0a427029ccab53738', 'request_id': 'req-1f508c1a-2501-4bf3-b993-466e4460a2c6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:11.769001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff0ee1edd3184df0a427029ccab53738', 'request_id': 'req-1f508c1a-2501-4bf3-b993-466e4460a2c6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:11.782641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff0ee1edd3184df0a427029ccab53738', 'request_id': 'req-1f508c1a-2501-4bf3-b993-466e4460a2c6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:11.907984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] The token cache is being invalidate because project ff0ee1edd3184df0a427029ccab53738 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:34:11.908432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:11.913243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1f508c1a-2501-4bf3-b993-466e4460a2c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:11.916273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1080/2120] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:34:11 2022] DELETE /identity/v3/projects/ff0ee1edd3184df0a427029ccab53738 => generated 0 bytes in 406 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:11.921228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:11.921748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:11.921827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:11.942320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:11.945436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:12.011624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1041/2121] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:12.026822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5f46-5017-478b-9853-5046f75577eb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.027300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5f46-5017-478b-9853-5046f75577eb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.028815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724e5f46-5017-478b-9853-5046f75577eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.050140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-724e5f46-5017-478b-9853-5046f75577eb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:12.051470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-724e5f46-5017-478b-9853-5046f75577eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:12.060192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1081/2122] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:12.064917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:12.076420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2abe57-96e2-419a-853c-957f3e973be3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:12.081835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:12.083071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.083991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.083991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.084504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:34:12.084894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:12.086476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:12.100868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1042/2123] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:34:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:34:12.105601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5f46-5017-478b-9853-5046f75577eb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:12.116049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724e5f46-5017-478b-9853-5046f75577eb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:12.121776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:12.122482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.122792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.123087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.123824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:12.126366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:12.244937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1082/2124] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:34:12 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:34:12.249005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:12.261282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6965817-f26d-452e-859e-232749b4a9f1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:12.324215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:12.324944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.325229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.325507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.326222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:12.328205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:12.432676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1043/2125] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:34:12 2022] POST /identity/v3/users => generated 292 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:12.446642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:12.457177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd3f4c6-c5da-43aa-9d96-fd99f1fc97b0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:12.464159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:12.464867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.465152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.465432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.466169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:12.467793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:12.479719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1083/2126] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:12.489290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:12.499958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d06a93f-a2da-4341-9c75-c168f8ce0843 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:12.505591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31577333-07f0-489b-b7b6-ac0fa7596dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:12.505981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31577333-07f0-489b-b7b6-ac0fa7596dde admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.506270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31577333-07f0-489b-b7b6-ac0fa7596dde admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.506552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31577333-07f0-489b-b7b6-ac0fa7596dde admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.507308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31577333-07f0-489b-b7b6-ac0fa7596dde admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:12.508980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31577333-07f0-489b-b7b6-ac0fa7596dde admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:12.520123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1044/2127] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:12 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 13:34:12.524780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:12.537170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb48ebf0-627c-4cf0-bfa6-88e80598bcd0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:12.543117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:12.544022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.544311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.544616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] PATH_INFO: `/v3/projects/3407c9665e0448e18fd87ec20b0843e4/users/701b374f2fcf437dad87cec756232547/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.562439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3407c9665e0448e18fd87ec20b0843e4, user_id=701b374f2fcf437dad87cec756232547, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:12.565185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:12.949403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1084/2128] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:12 2022] PUT /identity/v3/projects/3407c9665e0448e18fd87ec20b0843e4/users/701b374f2fcf437dad87cec756232547/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:12.954489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726b80-3a67-4606-94d2-c3bd082bb769 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:12.954961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726b80-3a67-4606-94d2-c3bd082bb769 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:12.955059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4726b80-3a67-4606-94d2-c3bd082bb769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:12.973052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f4726b80-3a67-4606-94d2-c3bd082bb769 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:12.974852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4726b80-3a67-4606-94d2-c3bd082bb769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:13.010761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:13.038431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1045/2129] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:13.048435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:13.050271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:13.109359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ff7b38b-b6ca-4e6a-a884-2e37b0ea7db7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:13.113110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0620a-0cf8-4354-b943-43d69f8eedcd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:13.113796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0620a-0cf8-4354-b943-43d69f8eedcd service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:13.114687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0620a-0cf8-4354-b943-43d69f8eedcd service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:13.114974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0620a-0cf8-4354-b943-43d69f8eedcd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:13.121376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:13.125106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0620a-0cf8-4354-b943-43d69f8eedcd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:13.126546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc0620a-0cf8-4354-b943-43d69f8eedcd service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:13.136462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1085/2130] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:34:17.118206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:17.119017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:17.119172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:17.139031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] MFA Rules not processed for user `740cc82ff2c04228b8ba32ecc034944f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:17.140264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:17.219341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1046/2131] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:17 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:20.154306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:20.155102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:20.155252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:20.200858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] MFA Rules not processed for user `701b374f2fcf437dad87cec756232547`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:20.202434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:20.286416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1086/2132] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:20.291843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:20.303955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b428484c-a8e6-423d-a25e-0d1c9b03964f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:20.311879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:20.312589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:20.312962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:20.313181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:20.313896 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:20.315842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:20.470836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1047/2133] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:34:20 2022] POST /identity/v3/projects => generated 383 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:20.476063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:20.494072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15637387-8e0c-4d2d-9ddd-f22674d81731 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:20.563266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:20.564201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:20.564752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:20.565157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:20.566099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:20.571029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:20.688043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1087/2134] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:34:20 2022] POST /identity/v3/users => generated 298 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:20.692725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:20.706342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63242e65-2643-4e4a-a1ba-043ae6677dc3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:20.714816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:20.724292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:20.724292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:20.724292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:20.724292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:20.724292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:20.733026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1048/2135] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:20.737489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:20.752605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65685ca7-1731-4361-b271-a92a8d4daccd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:20.761966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:20.762886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:20.763164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:20.763440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] PATH_INFO: `/v3/projects/8e381e1c71054b11a438eb349bf72a34/users/5696f2f99df545bebadfebb408e9c9bd/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:20.783326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e381e1c71054b11a438eb349bf72a34, user_id=5696f2f99df545bebadfebb408e9c9bd, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:20.788735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:20.913376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1088/2136] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:20 2022] PUT /identity/v3/projects/8e381e1c71054b11a438eb349bf72a34/users/5696f2f99df545bebadfebb408e9c9bd/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:20.968400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:20.968400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c35fcb30-cf5f-42f0-bc55-7717861442f9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:21.036518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:21.037462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:21.037811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:21.038106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:21.039521 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:21.042907 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:21.059573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1049/2137] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:20 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:21.063601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:21.078481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a4e546-3cf5-403e-8c27-5336cdaec315 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:21.086817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877d1b6-0f6d-4b20-a366-1cfaff6880c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:21.088598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877d1b6-0f6d-4b20-a366-1cfaff6880c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:21.089044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877d1b6-0f6d-4b20-a366-1cfaff6880c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:21.089270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877d1b6-0f6d-4b20-a366-1cfaff6880c9 admin admin] PATH_INFO: `/v3/projects/8e381e1c71054b11a438eb349bf72a34/users/5696f2f99df545bebadfebb408e9c9bd/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:21.091574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877d1b6-0f6d-4b20-a366-1cfaff6880c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8e381e1c71054b11a438eb349bf72a34, user_id=5696f2f99df545bebadfebb408e9c9bd, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:21.094407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877d1b6-0f6d-4b20-a366-1cfaff6880c9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:21.107806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1089/2138] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:21 2022] PUT /identity/v3/projects/8e381e1c71054b11a438eb349bf72a34/users/5696f2f99df545bebadfebb408e9c9bd/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:22.301881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:22.309958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:22.353322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0cfac5-30fd-49f0-bd5e-75e68b825f7d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:22.419778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:22.436464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:22.438059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:22.438543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:22.439437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] PATH_INFO: `/v3/users/ae7033cebdb0481ebf34126c19581cfa` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:22.441003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ae7033cebdb0481ebf34126c19581cfa)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:22.444041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:22.770478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ae7033cebdb0481ebf34126c19581cfa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:22.778977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ae7033cebdb0481ebf34126c19581cfa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:22.794137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ae7033cebdb0481ebf34126c19581cfa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:22.795017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ae7033cebdb0481ebf34126c19581cfa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:22.900136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ae7033cebdb0481ebf34126c19581cfa'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:22.911205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1050/2139] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:34:22 2022] DELETE /identity/v3/users/ae7033cebdb0481ebf34126c19581cfa => generated 0 bytes in 606 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:22.914938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:22.915802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:22.916267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:22.969468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:22.971319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:23.048195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:23.127220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1090/2140] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:23.139295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:23.141352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:23.221343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39729a0a-07ff-4ff9-a71f-b429a43cdbe5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:23.225830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355e80e7-6a2b-4467-a003-7ee9e3ddb0e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:23.226536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e80e7-6a2b-4467-a003-7ee9e3ddb0e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:23.226822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e80e7-6a2b-4467-a003-7ee9e3ddb0e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:23.227101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355e80e7-6a2b-4467-a003-7ee9e3ddb0e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:23.238324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e80e7-6a2b-4467-a003-7ee9e3ddb0e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:23.239322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355e80e7-6a2b-4467-a003-7ee9e3ddb0e8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:23.250969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1051/2141] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:34:24.000154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:24.002356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:24.032182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b47b841d-a142-42ec-9bc0-f5d10dd947d7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:24.043569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:24.044875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:24.045272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:24.045652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] PATH_INFO: `/v3/projects/c6c6ba32d8a84b9991fc9f0a7f14e463` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:24.047774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6c6ba32d8a84b9991fc9f0a7f14e463)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:24.050167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:24.263951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6c6ba32d8a84b9991fc9f0a7f14e463', 'request_id': 'req-1b23f0e9-6074-4298-91a5-07ef75078298'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:24.273215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6c6ba32d8a84b9991fc9f0a7f14e463', 'request_id': 'req-1b23f0e9-6074-4298-91a5-07ef75078298'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:24.289732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6c6ba32d8a84b9991fc9f0a7f14e463', 'request_id': 'req-1b23f0e9-6074-4298-91a5-07ef75078298'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:24.429349 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] The token cache is being invalidate because project c6c6ba32d8a84b9991fc9f0a7f14e463 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:34:24.433273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:24.433273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-1b23f0e9-6074-4298-91a5-07ef75078298 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:34:24.433273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1091/2142] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:34:23 2022] DELETE /identity/v3/projects/c6c6ba32d8a84b9991fc9f0a7f14e463 => generated 0 bytes in 433 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:24.437719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:24.438269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:24.438269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:24.462255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:24.463882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:24.594092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:24.594287 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:24.594590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:24.597709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1052/2143] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:24.628224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:24.630412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:24.645551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1092/2144] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:34:24.647697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:24.661297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2311de0-65a6-4e55-853c-42dc11484c18 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:24.669497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:24.670751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:24.671051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:24.671343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:24.672464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:34:24.672610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:24.675776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:24.687232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:24.700716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1053/2145] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:34:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:34:24.705565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:24.719121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b7166f-af31-49f6-94bd-5ba83b479abf None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:24.725053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:24.725762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:24.726451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:24.726689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:24.727645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:24.732045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:24.897543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1093/2146] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:34:24 2022] POST /identity/v3/projects => generated 373 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:24.901686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:24.916033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f3bba4-e975-4c5c-843c-a5884afaa813 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:25.060038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:25.061266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:25.061557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:25.061837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:25.066275 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:25.074997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:25.230615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1054/2147] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:34:24 2022] POST /identity/v3/users => generated 287 bytes in 329 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:34:25.234996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:25.252087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f83fd-85e2-463b-9f8a-4c3715c9fb92 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:25.263011 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:25.263979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:25.264265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:25.264551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:25.265320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:25.272710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:25.303998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1094/2148] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:25.306397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:25.338513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6921973e-b936-49dc-81e1-b56d645da679 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:25.355367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd8fcbf-75ce-46d4-9606-f18306ef0b03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:25.356070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fcbf-75ce-46d4-9606-f18306ef0b03 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:25.356360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fcbf-75ce-46d4-9606-f18306ef0b03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:25.356665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd8fcbf-75ce-46d4-9606-f18306ef0b03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:25.357756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8fcbf-75ce-46d4-9606-f18306ef0b03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:25.359122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd8fcbf-75ce-46d4-9606-f18306ef0b03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:25.388638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1055/2149] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:25 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:25.392555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:25.425069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4479cfc-f221-4728-97a3-5d449b728409 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:25.445549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:25.446979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:25.449598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:25.449854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] PATH_INFO: `/v3/projects/8441389e662041a1945ee7164ad17159/users/86ffd5c4601c4ccf90d09be72f41a33f/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:25.483213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8441389e662041a1945ee7164ad17159, user_id=86ffd5c4601c4ccf90d09be72f41a33f, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:25.489755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:25.595712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1095/2150] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:25 2022] PUT /identity/v3/projects/8441389e662041a1945ee7164ad17159/users/86ffd5c4601c4ccf90d09be72f41a33f/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:25.601225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e5d78-d7b9-428d-83b1-36d482875f6c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:25.601711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e5d78-d7b9-428d-83b1-36d482875f6c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:25.601820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81e5d78-d7b9-428d-83b1-36d482875f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:25.625400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b81e5d78-d7b9-428d-83b1-36d482875f6c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:25.626739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b81e5d78-d7b9-428d-83b1-36d482875f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:25.695492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1056/2151] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:25.705382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:25.707267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:25.788053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8199c76-2bc9-4e01-88ac-7a847771fb18 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:25.794047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:25.794731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:25.795020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:25.795297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:25.807560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:25.809337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:25.814008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:25.822470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1096/2152] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:31.661075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f626a7-b5f3-48db-8d74-19ad24d4010e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:31.661932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f626a7-b5f3-48db-8d74-19ad24d4010e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:31.662385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f626a7-b5f3-48db-8d74-19ad24d4010e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:31.710792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-71f626a7-b5f3-48db-8d74-19ad24d4010e None None] MFA Rules not processed for user `5696f2f99df545bebadfebb408e9c9bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:31.713034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71f626a7-b5f3-48db-8d74-19ad24d4010e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:31.794586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:31.813886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1057/2153] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:31 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:31.826896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:31.828600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:31.943281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c53881e-bb82-493b-9688-9f86b273b7d4 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:31.946913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:31.948046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:31.948046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:31.948256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:31.950304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:32.048820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:32.055820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:32.080633 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1097/2154] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:32.396309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f14c1-9a67-4ff8-95b6-f967337d4dd7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:32.396916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f14c1-9a67-4ff8-95b6-f967337d4dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:32.397423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c2f14c1-9a67-4ff8-95b6-f967337d4dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:32.492225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5c2f14c1-9a67-4ff8-95b6-f967337d4dd7 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:32.494338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c2f14c1-9a67-4ff8-95b6-f967337d4dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:32.611049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1058/2155] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:34:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:32.703066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:32.704995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:32.859639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906a5a31-05ec-4580-8e29-2f0f223ca68c service glance] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:32.875636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:32.876649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:32.877128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:32.877410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:32.894898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:32.897044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:32.918051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1098/2156] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:34:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 216 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:34:35.388712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:35.389341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:35.389459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:35.466263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] MFA Rules not processed for user `86ffd5c4601c4ccf90d09be72f41a33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:35.483384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:35.677689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:35.680819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1059/2157] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:35 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:35.683291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:35.702174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba52925c-3463-4f3a-99a3-c83b0c31d6c5 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:35.707443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:35.708209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:35.708642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:35.709029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:35.709857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:35.711939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:35.890548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1099/2158] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:34:35 2022] POST /identity/v3/projects => generated 373 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:34:35.900851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:35.919519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613c45ff-3301-4e6b-ad8d-e69248f5bd91 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:35.989182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:36.012639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:36.013765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:36.014247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:36.014787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:36.015783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:36.017808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:36.162078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1060/2159] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:34:35 2022] POST /identity/v3/users => generated 293 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:34:36.165825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:36.181099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9918d2bf-f47f-4c08-b83c-f2362915b4b2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:36.196450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:36.199147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:36.200144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:36.202743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:36.203900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:36.205847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:36.221250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1100/2160] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:36.228829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:36.247795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fce8b72d-1a5a-46c2-9128-610a052dc6e4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:36.258865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:36.259889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:36.260135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:36.260417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] PATH_INFO: `/v3/projects/726d8b40614546449ec3c3d3133530c7/users/e1fe58080a164ff8bd5f46006243a947/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:36.292551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=726d8b40614546449ec3c3d3133530c7, user_id=e1fe58080a164ff8bd5f46006243a947, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:36.298226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:36.392022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1061/2161] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:36 2022] PUT /identity/v3/projects/726d8b40614546449ec3c3d3133530c7/users/e1fe58080a164ff8bd5f46006243a947/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:36.394457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:36.414970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51be38ab-0881-4bad-9543-680feba7477a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:36.544548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507882a8-239f-4fb7-b2eb-29c1064f5e21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:36.545332 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507882a8-239f-4fb7-b2eb-29c1064f5e21 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:36.545603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507882a8-239f-4fb7-b2eb-29c1064f5e21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:36.545882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507882a8-239f-4fb7-b2eb-29c1064f5e21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:36.546922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507882a8-239f-4fb7-b2eb-29c1064f5e21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:36.548765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507882a8-239f-4fb7-b2eb-29c1064f5e21 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:36.570211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1101/2162] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:34:36 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:34:36.575683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:36.587510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-112115d4-8fd7-4380-8ad8-a2b26ee17df6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:36.595344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:36.597200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:36.597685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:36.598175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] PATH_INFO: `/v3/projects/726d8b40614546449ec3c3d3133530c7/users/e1fe58080a164ff8bd5f46006243a947/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:36.603019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] RBAC: Authorizing `identity:create_grant(project_id=726d8b40614546449ec3c3d3133530c7, user_id=e1fe58080a164ff8bd5f46006243a947, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:36.607147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:36.624823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1062/2163] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:34:36 2022] PUT /identity/v3/projects/726d8b40614546449ec3c3d3133530c7/users/e1fe58080a164ff8bd5f46006243a947/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:34:46.168886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:46.169510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:46.169510 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:46.210244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] MFA Rules not processed for user `e1fe58080a164ff8bd5f46006243a947`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:46.211642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:46.364428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1102/2164] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:46 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:46.380862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:46.383376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:46.554944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127b6c06-3ec0-4fcc-a67d-2a4e28079cca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:46.563882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b71443a-fc5b-4ac2-9659-b5885fd8bda0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:46.563882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71443a-fc5b-4ac2-9659-b5885fd8bda0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:46.563882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71443a-fc5b-4ac2-9659-b5885fd8bda0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:46.564236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b71443a-fc5b-4ac2-9659-b5885fd8bda0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:46.642797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71443a-fc5b-4ac2-9659-b5885fd8bda0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:46.644961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b71443a-fc5b-4ac2-9659-b5885fd8bda0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:46.657216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1063/2165] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:49.960057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:49.962133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:50.063198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:50.120377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b05098-8053-492b-9682-f2c8d351c3f3 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:50.126565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:50.127353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:50.127723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:50.128188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:50.130762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:50.145671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:50.150442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c23b28-184b-44c7-b9dd-f89e9de6159f service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:50.159227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1103/2166] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:34:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:34:50.646779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067d41d-f7d5-4e39-a465-a7f69c0d4460 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:50.647722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067d41d-f7d5-4e39-a465-a7f69c0d4460 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:50.648158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f067d41d-f7d5-4e39-a465-a7f69c0d4460 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:50.669638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f067d41d-f7d5-4e39-a465-a7f69c0d4460 None None] MFA Rules not processed for user `701b374f2fcf437dad87cec756232547`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:50.670975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f067d41d-f7d5-4e39-a465-a7f69c0d4460 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:50.683639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:34:50.735399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1064/2167] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:34:51.599060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e58104-28eb-4e66-b1d5-6c8ed6472d28 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:51.599060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e58104-28eb-4e66-b1d5-6c8ed6472d28 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:51.606526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1e58104-28eb-4e66-b1d5-6c8ed6472d28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:51.619403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e1e58104-28eb-4e66-b1d5-6c8ed6472d28 None None] MFA Rules not processed for user `86ffd5c4601c4ccf90d09be72f41a33f`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:51.620738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e1e58104-28eb-4e66-b1d5-6c8ed6472d28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:51.628118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1104/2168] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:34:52.329355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:52.329921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:52.330016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:52.348822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] MFA Rules not processed for user `740cc82ff2c04228b8ba32ecc034944f`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:52.359962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:52.409546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1065/2169] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:34:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:34:58.721402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa53a95-854a-4b33-b917-5ed18a09e768 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:58.722527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa53a95-854a-4b33-b917-5ed18a09e768 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:58.722988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa53a95-854a-4b33-b917-5ed18a09e768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:58.746728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-eaa53a95-854a-4b33-b917-5ed18a09e768 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:34:58.748453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eaa53a95-854a-4b33-b917-5ed18a09e768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:34:58.832908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1105/2170] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:34:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:34:58.844960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:34:58.948393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e316c-cc9d-4833-a567-e51e80011f9c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:34:58.954373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:34:58.955387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:34:58.955824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:34:58.956100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:34:58.980602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:34:58.982289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:34:59.002293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1066/2171] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:34:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:35:08.813703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=46 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:35:08.997693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=75 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:35:46.879106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2b3d-97aa-4f62-9662-18aa2eb99a30 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:35:46.880051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2b3d-97aa-4f62-9662-18aa2eb99a30 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:35:46.880492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976a2b3d-97aa-4f62-9662-18aa2eb99a30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:35:46.909157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-976a2b3d-97aa-4f62-9662-18aa2eb99a30 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:35:46.910996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-976a2b3d-97aa-4f62-9662-18aa2eb99a30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:35:46.919389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1106/2172] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:35:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:35:46.944681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:35:46.955024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19507037-878d-4a58-bb13-2b4c5fcfc72b service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:35:46.961938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:35:46.962790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:35:46.963224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:35:46.963798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:35:46.985147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:35:46.998519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:35:47.008449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1067/2173] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:35:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 13:35:56.718337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1588e-dbb0-4f59-8582-052ed4c47456 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:35:56.719361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1588e-dbb0-4f59-8582-052ed4c47456 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:35:56.719710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1588e-dbb0-4f59-8582-052ed4c47456 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:35:56.742405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3ef1588e-dbb0-4f59-8582-052ed4c47456 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:35:56.744188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3ef1588e-dbb0-4f59-8582-052ed4c47456 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:35:56.754507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1107/2174] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:35:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:35:56.774115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:35:56.790598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c241327b-5590-439d-b8ce-099b0c57af54 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:35:56.796889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:35:56.797780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:35:56.798360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:35:56.799393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:35:56.810126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:35:56.812011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:35:56.821905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1068/2175] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:35:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:36:02.214454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46030-5d1b-45a9-a4c5-0b94d50dae76 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:02.214454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46030-5d1b-45a9-a4c5-0b94d50dae76 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:02.214454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a46030-5d1b-45a9-a4c5-0b94d50dae76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:02.242428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-58a46030-5d1b-45a9-a4c5-0b94d50dae76 None None] MFA Rules not processed for user `cf30fa512142489a8dea40ce426a5f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:02.245639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-58a46030-5d1b-45a9-a4c5-0b94d50dae76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:02.320683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1108/2176] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:36:02 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:06.817931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:08.415607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:08.417525 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:08.622191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32484282-93d0-4cd0-a795-dba3639563b1 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:08.629763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4bf0c5-29f0-42d1-9cd1-ee8b3c2574b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:08.630655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bf0c5-29f0-42d1-9cd1-ee8b3c2574b3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:08.630942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bf0c5-29f0-42d1-9cd1-ee8b3c2574b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:08.631281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4bf0c5-29f0-42d1-9cd1-ee8b3c2574b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:08.757481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bf0c5-29f0-42d1-9cd1-ee8b3c2574b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:08.759105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4bf0c5-29f0-42d1-9cd1-ee8b3c2574b3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:08.776601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1069/2177] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:08 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:12.312930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:15.581173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e66a-0d5e-4697-a850-82a1f0300eb7 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:15.582220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e66a-0d5e-4697-a850-82a1f0300eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:15.582220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d0e66a-0d5e-4697-a850-82a1f0300eb7 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:15.583028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1109/2178] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:36:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:36:15.594808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18bc04-35a0-4954-92f1-0ad441063781 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:15.596117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18bc04-35a0-4954-92f1-0ad441063781 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:15.596117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de18bc04-35a0-4954-92f1-0ad441063781 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:15.648987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-de18bc04-35a0-4954-92f1-0ad441063781 None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:15.650280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-de18bc04-35a0-4954-92f1-0ad441063781 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:15.685208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1070/2179] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:36:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:36:15.698896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e66a-0d5e-4697-a850-82a1f0300eb7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:15.701210 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b4d0e66a-0d5e-4697-a850-82a1f0300eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:15.792261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d0e66a-0d5e-4697-a850-82a1f0300eb7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:15.796305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f53040-39fb-491e-8ef0-2c33e89ac16a service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:15.797006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53040-39fb-491e-8ef0-2c33e89ac16a service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:15.797292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53040-39fb-491e-8ef0-2c33e89ac16a service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:15.797737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f53040-39fb-491e-8ef0-2c33e89ac16a service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:15.817799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f53040-39fb-491e-8ef0-2c33e89ac16a service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:15.819775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f53040-39fb-491e-8ef0-2c33e89ac16a service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:15.832673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1110/2180] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:16.085179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1868b-caa8-4c0a-a37a-f0cd17d115a8 None None] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:16.090123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1868b-caa8-4c0a-a37a-f0cd17d115a8 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:16.090734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d1868b-caa8-4c0a-a37a-f0cd17d115a8 None None] PATH_INFO: `/` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:16.094255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1071/2181] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:36:16 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:36:16.105596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a3f12-bcd9-4a22-85e6-7488f75a8d53 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:16.106188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a3f12-bcd9-4a22-85e6-7488f75a8d53 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:16.106627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a3f12-bcd9-4a22-85e6-7488f75a8d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:16.138897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e59a3f12-bcd9-4a22-85e6-7488f75a8d53 None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:16.141049 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e59a3f12-bcd9-4a22-85e6-7488f75a8d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:16.152342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1111/2182] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:36:16 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:16.166663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1868b-caa8-4c0a-a37a-f0cd17d115a8 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:16.182569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d1868b-caa8-4c0a-a37a-f0cd17d115a8 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:16.197054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42cc5bc-b71f-4a60-a127-1ccb141d6510 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:16.197837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cc5bc-b71f-4a60-a127-1ccb141d6510 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:16.200850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cc5bc-b71f-4a60-a127-1ccb141d6510 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:16.205806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42cc5bc-b71f-4a60-a127-1ccb141d6510 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:16.213268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cc5bc-b71f-4a60-a127-1ccb141d6510 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:16.215253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42cc5bc-b71f-4a60-a127-1ccb141d6510 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:16.228215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1072/2183] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:16.382184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f165c0-8546-49ab-9ba6-79937cd4b005 None None] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:16.382562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f165c0-8546-49ab-9ba6-79937cd4b005 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:16.382782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f165c0-8546-49ab-9ba6-79937cd4b005 None None] PATH_INFO: `/` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:16.384301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1112/2184] 10.222.0.70 () {54 vars in 921 bytes} [Mon May 23 13:36:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 23 13:36:16.390343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0e3b3-43cf-41db-8793-a31bd8448198 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:16.390630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0e3b3-43cf-41db-8793-a31bd8448198 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:16.390916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed0e3b3-43cf-41db-8793-a31bd8448198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:16.423936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7ed0e3b3-43cf-41db-8793-a31bd8448198 None None] MFA Rules not processed for user `9cba93afd3344df298ba353d9994ac37`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:16.423936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ed0e3b3-43cf-41db-8793-a31bd8448198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:16.441527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1073/2185] 10.222.0.70 () {58 vars in 1018 bytes} [Mon May 23 13:36:16 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:16.452324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f165c0-8546-49ab-9ba6-79937cd4b005 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:16.467964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f165c0-8546-49ab-9ba6-79937cd4b005 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:16.474270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:16.475248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:16.475575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:16.475945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:16.502886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:16.504198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:16.521773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1113/2186] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:36:20.826390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:20.826390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:20.830636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:20.870028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:20.871480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:20.881599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1074/2187] 10.222.0.70 () {58 vars in 967 bytes} [Mon May 23 13:36:20 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:20.900345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:20.914250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34eccfb-89c1-435d-8d3c-fb033dd046cf service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:20.919861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab5df495-e686-45d0-9a62-534dbfd8bc44 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:20.921723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df495-e686-45d0-9a62-534dbfd8bc44 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:20.922015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df495-e686-45d0-9a62-534dbfd8bc44 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:20.922303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab5df495-e686-45d0-9a62-534dbfd8bc44 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:20.935137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5df495-e686-45d0-9a62-534dbfd8bc44 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:20.936873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab5df495-e686-45d0-9a62-534dbfd8bc44 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:20.964722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1114/2188] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:36:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:36:21.662961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:21.672214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea431a7-bdfd-4824-a0bd-4c2099125436 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:21.676484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:21.677154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:21.677468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:21.677810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:21.688943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:21.690869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:21.701440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1075/2189] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:21 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:27.537086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:27.537597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:27.537720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:27.555870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] MFA Rules not processed for user `659caf32fa614f64bcd780a264674e33`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:27.557120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:27.625013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1115/2190] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:36:27 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:29.093885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:29.099857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:29.172438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:29.212642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81bfc3a-7969-4063-9e86-297d2a5ec96c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:29.225530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:29.226519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:29.226809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:29.227090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] PATH_INFO: `/v3/users/740cc82ff2c04228b8ba32ecc034944f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:29.228463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] RBAC: Authorizing `identity:delete_user(user_id=740cc82ff2c04228b8ba32ecc034944f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:29.231111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:29.494018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '740cc82ff2c04228b8ba32ecc034944f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:29.502833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '740cc82ff2c04228b8ba32ecc034944f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:29.516208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '740cc82ff2c04228b8ba32ecc034944f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:29.517054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '740cc82ff2c04228b8ba32ecc034944f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:29.788346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3d9d525c-0eb0-4631-aa63-2e0aeee66832 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '740cc82ff2c04228b8ba32ecc034944f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:29.795646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1076/2191] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:36:29 2022] DELETE /identity/v3/users/740cc82ff2c04228b8ba32ecc034944f => generated 0 bytes in 703 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:36:30.095422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:30.108293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee06f09-b818-4640-b9ad-748e470bb530 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:30.168529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:30.169425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:30.169710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:30.169983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] PATH_INFO: `/v3/projects/b3808ebec9df4a4c937ab961f3683dda` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:30.171356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b3808ebec9df4a4c937ab961f3683dda)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:30.173700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:30.193246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:30.362977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b3808ebec9df4a4c937ab961f3683dda', 'request_id': 'req-5c9c6837-734c-42d1-99b0-5e60673aa5c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:30.370066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b3808ebec9df4a4c937ab961f3683dda', 'request_id': 'req-5c9c6837-734c-42d1-99b0-5e60673aa5c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:30.375641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b3808ebec9df4a4c937ab961f3683dda', 'request_id': 'req-5c9c6837-734c-42d1-99b0-5e60673aa5c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:30.511178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] The token cache is being invalidate because project b3808ebec9df4a4c937ab961f3683dda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:36:30.511178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:30.511178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5c9c6837-734c-42d1-99b0-5e60673aa5c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:36:30.512265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1116/2192] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:36:30 2022] DELETE /identity/v3/projects/b3808ebec9df4a4c937ab961f3683dda => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:36:30.518565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:30.519085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:30.519189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:30.543751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:30.545043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:30.611945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1077/2193] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:30.619995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:30.620338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:30.620558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:30.639843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:30.643524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:30.649625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1117/2194] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:36:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:30.653846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:30.664418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97f676e4-b521-4c21-8257-5e077bf9f8ea None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:30.670484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:30.671165 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:30.671451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:30.671731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:30.672480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:36:30.673463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:30.675268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:30.687817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1078/2195] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:36:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:36:30.693095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:30.715263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e8c4b3-bdbc-432f-a37e-88966cad8929 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:30.722936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:30.723931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:30.724386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:30.724945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:30.725816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:30.728828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:30.894086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1118/2196] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:36:30 2022] POST /identity/v3/projects => generated 363 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:36:30.894717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:30.906729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7247d6-c232-464e-b9cf-526f842fe74f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:30.971910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:30.978459 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:30.979234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:30.979594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:30.979899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:30.980715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:30.982666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:31.126547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1079/2197] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:36:30 2022] POST /identity/v3/users => generated 282 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 13:36:31.133295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:31.144498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633eca28-2362-4a26-9a46-d08dbf1f8517 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:31.151272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:31.151740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:31.152027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:31.152314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:31.153586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:31.154688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:31.170042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1119/2198] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:36:31.176020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:31.186721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddef301-0f41-4dbf-b3b2-a4e502cdab41 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:31.192865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8fcbc0-a4a8-4581-a81f-393f5d5bfc59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:31.193562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcbc0-a4a8-4581-a81f-393f5d5bfc59 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:31.193917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcbc0-a4a8-4581-a81f-393f5d5bfc59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:31.194272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8fcbc0-a4a8-4581-a81f-393f5d5bfc59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:31.195491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcbc0-a4a8-4581-a81f-393f5d5bfc59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:31.197218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b8fcbc0-a4a8-4581-a81f-393f5d5bfc59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:31.210008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1080/2199] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:36:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:36:31.214190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:31.226348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18bf6410-b5c4-4b04-83c6-cd9f933bb5e2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:31.231734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:31.232884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:31.233058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:31.233337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] PATH_INFO: `/v3/projects/a8786346311c466485a362be5942969f/users/5f152cfc366941fdb8176a6ce0736823/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:31.258749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] RBAC: Authorizing `identity:create_grant(project_id=a8786346311c466485a362be5942969f, user_id=5f152cfc366941fdb8176a6ce0736823, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:31.261478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:31.384247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1120/2200] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:36:31 2022] PUT /identity/v3/projects/a8786346311c466485a362be5942969f/users/5f152cfc366941fdb8176a6ce0736823/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:36:31.396285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f560780-2558-4029-987a-860012bda2df None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:31.396604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f560780-2558-4029-987a-860012bda2df None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:31.397020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f560780-2558-4029-987a-860012bda2df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:31.426958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1f560780-2558-4029-987a-860012bda2df None None] MFA Rules not processed for user `5f152cfc366941fdb8176a6ce0736823`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:31.428454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1f560780-2558-4029-987a-860012bda2df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:31.511030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1081/2201] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:36:31 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:31.533230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:31.533230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:31.598638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba87c301-0a67-43d1-bfd5-df379083272b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:31.602328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:31.603087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:31.603444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:31.603786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:31.620544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:31.622188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:31.647247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1121/2202] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:31 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:34.343269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:34.343782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:34.343881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:34.362577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:36:34.363792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:34.509497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1082/2203] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 13:36:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:36:34.528365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:34.529823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:34.626346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:34.654112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923bfa3b-364e-4026-b139-875072634db8 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:34.660879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:34.661705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:34.661994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:34.662271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:34.672488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:34.675423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:34.687077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1122/2204] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:36:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:36:36.126590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:36.126590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:36.228666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992db049-aa25-45fa-b589-f1ed93b718e6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:36.232071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:36.232908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:36.233792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:36.233792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:36.236535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:36.275905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:36.339033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:36.340505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:36.354095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1083/2205] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:36.886206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:36.889189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:36.910732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afb9977-9df7-4cec-bf52-a6e5164f4106 service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:36.915982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:36.917215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:36.917831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:36.918456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:36.921290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:36.938310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:36.941382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:36.966948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1123/2206] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:46.367590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=22 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:46.949048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=18 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:36:52.371448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:52.372999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:52.387618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccad82a-4d74-464d-bd5d-dcb97f736ba6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:52.393436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:52.394094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:52.394379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:52.394656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:52.396437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:52.471443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:52.473132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:52.482453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1084/2207] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:56.927415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:56.937228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79c6e09-b044-4e49-95ef-b89d367c1d4a service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:56.942559 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:56.942953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:56.943203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:56.943484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:56.945525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:36:56.954840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:36:56.956304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:56.957974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6134cc-d817-4e67-b0a4-fba7034afcdb service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:56.965175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f23684e-1924-4698-91b8-2dda8355d2e5 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:36:56.966519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1124/2208] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:36:56.969156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:36:56.969819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:36:56.970111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:36:56.970393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:36:56.979070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:36:56.980539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:36:56.992994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1085/2209] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:36:56 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:37:06.962785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:37:06.988031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=34 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:37:17.726312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:37:17.726858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:37:17.726858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:37:17.745816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:37:17.747001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:37:17.754441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1125/2210] 10.222.0.70 () {58 vars in 967 bytes} [Mon May 23 13:37:17 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:37:17.777778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:37:17.790475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d08f69-be56-4e7c-a139-3b9c5f5f8484 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:37:17.797466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:37:17.798176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:37:17.798509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:37:17.798786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:37:17.808340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:37:17.810000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:37:17.828751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1086/2211] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 13:37:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 13:37:23.768066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:37:23.769604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:37:23.785764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c6ec7d-2cf6-4ae3-8fff-b8e2f44efb14 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:37:23.791189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:37:23.791832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:37:23.792118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:37:23.792396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:37:23.794157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:37:23.853168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:37:23.854786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:37:23.864527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1126/2212] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:37:23 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:37:27.815248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:37:33.860229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=31 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:38:09.801167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:38:09.813701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832b60e0-546d-4e3c-a8e2-3da879e44bc4 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:38:09.819566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:38:09.820253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:38:09.820540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:38:09.820853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:38:09.823039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:38:09.897866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:38:09.899403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:38:09.909956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1087/2213] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:38:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:38:19.905337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:38:35.532470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:38:35.534069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:38:35.548313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1f94d5-230e-483a-89e9-1dda7d435f57 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:38:35.552199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:38:35.552961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:38:35.553253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:38:35.553534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:38:35.555451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:38:35.634701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:38:35.636212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:38:35.648288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1127/2214] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:38:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:38:45.643958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:38:53.642065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:38:53.657056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a233104-7dd0-42a0-967a-9553658830f6 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:38:53.663611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:38:53.664322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:38:53.664643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:38:53.664925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:38:53.666868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:38:53.742369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:38:53.744046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e02777-e099-4636-9718-a89a94f729a8 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:38:53.755634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1088/2215] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:38:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:38:58.398237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:38:58.413467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24780182-2260-476b-92ba-92418fe6112f service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:38:58.423670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:38:58.424863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:38:58.425357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:38:58.425843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:38:58.430115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:38:58.455338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:38:58.459088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:38:58.484682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1128/2216] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:38:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:39:03.750792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:39:05.806230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:39:05.807100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:39:05.807234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:39:05.836402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] MFA Rules not processed for user `af29f227080147a1a157f88842d39483`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:39:05.838631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:39:05.851696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1089/2217] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:39:05 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:39:08.473905 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:39:15.828299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:39:46.886058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:39:46.906815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7398be4-2f4e-419e-b881-587dd78e9279 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:39:46.917694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:39:46.918223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:39:46.919000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:39:46.919000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:39:46.919000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:39:46.921156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:39:46.936541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:39:46.936883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca262f3-5146-4343-9cf2-5496cda0bed7 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:39:46.939562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:39:46.944514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:39:46.945320 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:39:46.945665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:39:46.946005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:39:46.948958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:39:46.958606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1129/2218] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:39:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:39:47.060775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:39:47.063144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:39:47.076018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1090/2219] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:39:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:39:55.581590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:39:55.595446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f065ec8a-f941-4053-9c08-59d21986d625 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:39:55.602661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:39:55.603410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:39:55.603823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:39:55.604164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:39:55.608825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:39:55.705836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:39:55.708074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:39:55.724236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1130/2220] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:39:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:39:57.069840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=28 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:39:58.502007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:39:58.514910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee686ba9-4001-43ac-bf00-ad7f553fe5f0 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:39:58.520994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:39:58.521943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:39:58.522225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:39:58.522502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:39:58.524532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:39:58.539018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:39:58.541686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:39:58.553427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1091/2221] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:39:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:40:05.716217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=34 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:40:08.548521 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:40:32.183323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:40:32.184865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:40:32.278455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9770f441-c62f-4fac-beca-614b0143d5de service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:40:32.283720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:40:32.284398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:40:32.284715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:40:32.284987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:40:32.286989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:40:32.300910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:40:32.302819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:40:32.315194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1131/2222] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:40:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:40:42.310156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=32 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:41:44.245933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:41:44.259392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63470a71-0e90-4a70-9fd7-0973dcaf1413 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:41:44.265809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:41:44.266949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:41:44.267461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:41:44.267962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:41:44.288722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:41:44.291843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:41:44.313911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1092/2223] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:41:44.451074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:41:44.461328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b00e9e-a3b6-4755-bc22-32b2cd8cd8d2 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:41:44.465483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:41:44.466277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:41:44.466591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:41:44.466915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:41:44.479286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:41:44.480836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:41:44.494555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1132/2224] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:41:44.618956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:41:44.631850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad8335b-9980-4185-9ab3-ac636de4bfce service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:41:44.638269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:41:44.639010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:41:44.639292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:41:44.639566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:41:44.650823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:41:44.652289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:41:44.666609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1093/2225] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:41:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:41:54.488492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:41:54.658311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=12 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:42:30.219073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:42:30.230644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d44573-2298-4a29-8438-1c9d2c7ea63e service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:42:30.237189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:42:30.238301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:42:30.238804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:42:30.239281 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:42:30.250786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:42:30.253050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:42:30.293975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1133/2226] 10.222.0.70 () {60 vars in 1396 bytes} [Mon May 23 13:42:30 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:42:35.646694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:42:35.657232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c72691f-aa69-44e7-8bcd-b5930913f15d service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:42:35.662096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:42:35.662917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:42:35.663199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:42:35.663494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:42:35.680352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:42:35.681914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:42:35.693399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1094/2227] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:42:35 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:42:40.290975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=13 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:42:45.689676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:43:45.148485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:43:45.196843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c147795f-a56a-44cb-99ec-1334cecd6c5d service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:43:45.201077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:43:45.201770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:43:45.202057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:43:45.202337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:43:45.230451 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:43:45.232974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:43:45.270468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1134/2228] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:43:45.402155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:43:45.403295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:43:45.419546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b4020-7987-4938-8945-00766de3f7b8 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:43:45.426479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:43:45.427577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:43:45.428072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:43:45.428552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:43:45.440420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e5097d-379a-4bd5-bb23-f89f66045859 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:43:45.444277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:43:45.446979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:43:45.449343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:43:45.450414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:43:45.451509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:43:45.451963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:43:45.465704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:43:45.468678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:43:45.497978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1135/2229] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:43:45.523600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1095/2230] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:43:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:43:55.492743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=44 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:43:55.518565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=21 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:45:37.045981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:37.047968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:37.112486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b099e-99a2-4276-8e15-a521861329e8 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:37.118473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:37.119673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:37.119963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:37.120247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:37.130441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:37.140908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:37.144543 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1136/2231] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:45:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:45:45.789634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:45.801787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af705f45-772d-4638-b075-90bff99fe30b service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:45.809037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:45.810829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:45.811341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:45.811819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:45.828941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:45.831879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:45.884095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1096/2232] 10.222.0.70 () {60 vars in 1396 bytes} [Mon May 23 13:45:45 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:45:45.945892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:45.986686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10b741d4-75a3-415f-9084-04d39234c86c service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:45.992792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:45.993360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:45.993644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:45.993922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:46.006734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:46.006734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:46.037405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1137/2233] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:45:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:45:46.212154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:46.226927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225979d4-67bb-478b-9b1a-a87879cdfbcf service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:46.233122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda904b7-b4db-48aa-87ec-6daaa0a1b3ae service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:46.233841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda904b7-b4db-48aa-87ec-6daaa0a1b3ae service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:46.234155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda904b7-b4db-48aa-87ec-6daaa0a1b3ae service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:46.234430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda904b7-b4db-48aa-87ec-6daaa0a1b3ae service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:46.243393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda904b7-b4db-48aa-87ec-6daaa0a1b3ae service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:46.245234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda904b7-b4db-48aa-87ec-6daaa0a1b3ae service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:46.276768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1097/2234] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:45:46 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:45:46.289360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:46.411151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8534d71-1e56-4026-9790-f36fbdb7e06c service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:46.415967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:46.416877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:46.417165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:46.417435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] PATH_INFO: `/v3/users/5f152cfc366941fdb8176a6ce0736823` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:46.418663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f152cfc366941fdb8176a6ce0736823)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:46.420706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:46.662230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f152cfc366941fdb8176a6ce0736823'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:46.669997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f152cfc366941fdb8176a6ce0736823'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:46.669997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f152cfc366941fdb8176a6ce0736823'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:46.672464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:45:46.677129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f152cfc366941fdb8176a6ce0736823'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:46.685428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f152cfc366941fdb8176a6ce0736823'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:46.877822 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1138/2235] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:45:46 2022] DELETE /identity/v3/users/5f152cfc366941fdb8176a6ce0736823 => generated 0 bytes in 589 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:45:46.882762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:46.910410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:46.910410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:46.929996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:45:46.931237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:46.997076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1098/2236] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:45:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:45:47.007814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:47.009389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:47.095546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7271c6ca-5c90-4306-88ec-2f4dda2d249e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:47.099856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bfafd93-ab28-43b7-8127-ce2b0d79023b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:47.101883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfafd93-ab28-43b7-8127-ce2b0d79023b service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:47.102222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfafd93-ab28-43b7-8127-ce2b0d79023b service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:47.102483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bfafd93-ab28-43b7-8127-ce2b0d79023b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:47.114485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfafd93-ab28-43b7-8127-ce2b0d79023b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:47.117004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bfafd93-ab28-43b7-8127-ce2b0d79023b service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:47.127080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1139/2237] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:45:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:45:47.713192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:47.717515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:47.727893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca6c1c2-5040-41be-a504-f2699c794afb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:47.732982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:47.733870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:47.734145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:47.735159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] PATH_INFO: `/v3/projects/a8786346311c466485a362be5942969f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:47.736749 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a8786346311c466485a362be5942969f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:47.738887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:47.903263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a8786346311c466485a362be5942969f', 'request_id': 'req-ba07684a-82d8-421d-968d-513d3f224ed2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:47.910033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a8786346311c466485a362be5942969f', 'request_id': 'req-ba07684a-82d8-421d-968d-513d3f224ed2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:47.917355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a8786346311c466485a362be5942969f', 'request_id': 'req-ba07684a-82d8-421d-968d-513d3f224ed2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:48.003225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] The token cache is being invalidate because project a8786346311c466485a362be5942969f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:45:48.003487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:48.004230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ba07684a-82d8-421d-968d-513d3f224ed2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:45:48.005928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1099/2238] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:45:47 2022] DELETE /identity/v3/projects/a8786346311c466485a362be5942969f => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:45:48.012195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.012195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.012961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.066886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:45:48.068132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:48.136763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1140/2239] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 13:45:48.144843 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.145407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.145407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.166412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:45:48.167626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:48.176844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1100/2240] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:45:48.181797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:48.193460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e920bcd-2d1e-4306-968f-d452880d14c4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:48.198551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:48.199236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.199511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.199791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.200538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 13:45:48.200882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:48.202496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:48.215889 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1141/2241] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 13:45:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 13:45:48.220365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:48.224841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:45:48.231594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18cd5a06-f631-4d30-891f-1d97ec26964c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:48.236362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:48.240213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.240213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.240213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.240213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:48.240213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:48.422917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1101/2242] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 13:45:48 2022] POST /identity/v3/projects => generated 375 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:45:48.427407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:48.439060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeed91a2-6533-45c4-ac35-6e4514c1e625 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:48.462137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:45:48.501793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:48.502484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.502771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.503058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.503771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:48.505796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:48.638569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1142/2243] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 13:45:48 2022] POST /identity/v3/users => generated 288 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 13:45:48.643352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:48.662281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d14e705-794e-4581-97d3-df146ea93c1d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:48.667626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:48.668370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.668684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.668964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.669716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:48.671363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:48.702476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1102/2244] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:45:48.702787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:48.711655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7513f1dd-0384-4897-827a-849e9b96cada admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:48.716522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32ea6de-01a0-45dd-83c9-cb1b9e3bdea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:48.717362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ea6de-01a0-45dd-83c9-cb1b9e3bdea2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.717657 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ea6de-01a0-45dd-83c9-cb1b9e3bdea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.717942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32ea6de-01a0-45dd-83c9-cb1b9e3bdea2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.718692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ea6de-01a0-45dd-83c9-cb1b9e3bdea2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:48.720324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32ea6de-01a0-45dd-83c9-cb1b9e3bdea2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:48.732954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1143/2245] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 13:45:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 13:45:48.738291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:45:48.754887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a7689e-842a-476a-a598-d1cadfc6d27e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:45:48.763804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:45:48.765243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.765758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.766152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] PATH_INFO: `/v3/projects/7d7f6067714a447c9564063d308e29f6/users/f0b992eb36b64b69906559fcbcc322b9/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.782062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7d7f6067714a447c9564063d308e29f6, user_id=f0b992eb36b64b69906559fcbcc322b9, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:45:48.784766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:45:48.883734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1103/2246] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 13:45:48 2022] PUT /identity/v3/projects/7d7f6067714a447c9564063d308e29f6/users/f0b992eb36b64b69906559fcbcc322b9/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:45:48.894236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:45:48.897176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:45:48.897176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:45:48.931787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] MFA Rules not processed for user `f0b992eb36b64b69906559fcbcc322b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:45:48.933088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:45:49.017340 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1144/2247] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:45:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:45:58.878921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=27,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:45:59.013515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=56,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:46:31.142666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:46:31.146311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:46:31.240323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a112e54-9cc3-4d7d-a51b-ad57db7a1fb7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:46:31.244494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:46:31.245202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:46:31.245493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:46:31.245768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:46:31.255782 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:46:31.257390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:46:31.267601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1104/2248] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:46:31 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:46:41.262828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=31 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:47:38.730866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:47:38.732434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:47:38.793807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d6d3-035e-4e77-abb7-8907b8111b2c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:47:38.797485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:47:38.799302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:47:38.799602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:47:38.799882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:47:38.801760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:47:38.812322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:47:38.814507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:47:38.823343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1145/2249] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:47:38 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:47:46.381025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:47:46.382468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:47:46.394620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a2858e-8e25-4a36-a1e2-13a85e24e476 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:47:46.398588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:47:46.399241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:47:46.399527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:47:46.399807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:47:46.401602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:47:46.489413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:47:46.490798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:47:46.502217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1105/2250] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:47:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:47:46.509369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:47:46.520374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed1c93-a5c4-4eec-b9a6-408a98c65994 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:47:46.525606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:47:46.526546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:47:46.526972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:47:46.527384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:47:46.529983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:47:46.609344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:47:46.611391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:47:46.621234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1146/2251] 10.222.0.70 () {60 vars in 1396 bytes} [Mon May 23 13:47:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:47:46.672323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:47:46.682138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fecbd5a-1264-478f-a734-1e83986a1817 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:47:46.685966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:47:46.686634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:47:46.686919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:47:46.687199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:47:46.689153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:47:46.704595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:47:46.706061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:47:46.717762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1106/2252] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:47:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:47:56.616452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=59 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:47:56.712531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=45 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:49:46.924535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:49:46.924535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:46.943493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13486cf-025f-4323-b950-2c7c45abc847 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:49:46.946892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:49:46.947572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:49:46.947901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:49:46.948138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:49:46.950168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:46.956096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:49:46.966508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4f9dbb-1fb1-4f27-a899-1e430b0c2877 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:49:46.972119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:49:46.973216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:49:46.973216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:49:46.973458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:49:46.975318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:47.066437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:49:47.069686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:49:47.079124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:49:47.080650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:49:47.088633 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1147/2253] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:49:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 169 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:49:47.093537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1107/2254] 10.222.0.70 () {60 vars in 1396 bytes} [Mon May 23 13:49:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:49:47.506014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:49:47.507495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:47.524486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65844e85-9e5c-4b05-955b-348cc159e0b7 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:49:47.533036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:49:47.533973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:49:47.534443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:49:47.534843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:49:47.537372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:47.642025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:49:47.643587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:49:47.655711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1148/2255] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:49:47 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:49:50.484874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:49:50.511448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cd718-86b0-4598-9ed3-56af4dbe9192 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:49:50.518551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:49:50.519593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:49:50.519878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:49:50.520438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:49:50.523147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:50.551996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:49:50.553883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:49:50.576637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1108/2256] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:49:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:49:52.007171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:49:52.020952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8670ea8f-a18e-4b3c-ac96-352cbb20e354 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:49:52.030429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:49:52.031501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:49:52.031979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:49:52.032485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:49:52.035520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:49:52.051006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:49:52.052823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:49:52.066581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1149/2257] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:49:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:50:00.566403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=39 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:50:02.059915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=72 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:50:43.743532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:50:43.744980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:50:43.817263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2131bf28-c976-4fd0-8b7a-68fa03f69d24 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:50:43.822321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:50:43.822980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:50:43.823267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:50:43.823547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:50:43.826322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:50:43.838560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:50:43.840964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:50:43.851279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1109/2258] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:50:43 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:50:46.426639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:50:46.438115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c64cb-ecdc-4990-813f-aa13fa62c1c5 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:50:46.443591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:50:46.444241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:50:46.444528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:50:46.444842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:50:46.446632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:50:46.523306 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:50:46.524964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:50:46.535842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1150/2259] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:50:46 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:50:53.847076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:50:56.530930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=33 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:51:52.304671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:51:52.315571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590550d3-e54b-4ea8-884b-2a095eec8088 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:51:52.319235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:51:52.319885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:51:52.320170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:51:52.320454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:51:52.323745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:51:52.407276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:51:52.408767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:51:52.417879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1110/2260] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:51:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:51:52.466676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:51:52.476066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f4d22-3f76-45ac-a09c-5bef24af7c4f service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:51:52.480016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:51:52.480710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:51:52.480994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:51:52.481267 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:51:52.490150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:51:52.491452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:51:52.502348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1151/2261] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:51:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:51:52.642442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:51:52.651989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-115bd8bb-11e2-4bb7-b1f2-ed3d8b6d990a service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:51:52.655955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:51:52.656644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:51:52.656930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:51:52.657210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:51:52.659169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:51:52.737908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:51:52.739280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:51:52.749817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1111/2262] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:51:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:52:02.497172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:52:02.745469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=66 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:52:45.444103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:52:45.446353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:52:45.459369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8131308b-9d39-4963-82cb-59a6fca6b9e5 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:52:45.464430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:52:45.465133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:52:45.465414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:52:45.465689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:52:45.473670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:52:45.475102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:52:45.484292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1152/2263] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:52:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:52:46.777000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:52:46.790664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62203639-7e28-4844-b4d3-44ae047de0cc service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:52:46.796209 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:52:46.796891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:52:46.797175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:52:46.797448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:52:46.806068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:52:46.807659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:52:46.836997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1112/2264] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:52:46 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:52:47.169095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:52:47.178228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e570a048-2a10-40df-86b2-20f6b2ae28c5 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:52:47.182881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9189af8d-ea28-4591-bed7-28467ad0841e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:52:47.184053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189af8d-ea28-4591-bed7-28467ad0841e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:52:47.184353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189af8d-ea28-4591-bed7-28467ad0841e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:52:47.185392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9189af8d-ea28-4591-bed7-28467ad0841e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:52:47.199573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189af8d-ea28-4591-bed7-28467ad0841e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:52:47.201093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9189af8d-ea28-4591-bed7-28467ad0841e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:52:47.213034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1153/2265] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:52:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:52:56.832849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=13 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:52:57.207723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=12 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:54:33.572732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:54:33.689730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3905293-3308-4f4a-9963-1326209ff00c service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:54:33.696078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:54:33.697042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:54:33.697473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:54:33.697759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] PATH_INFO: `/v3/users/f0b992eb36b64b69906559fcbcc322b9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:54:33.699047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f0b992eb36b64b69906559fcbcc322b9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:54:33.701102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:54:33.878169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f0b992eb36b64b69906559fcbcc322b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:33.885673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f0b992eb36b64b69906559fcbcc322b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:33.901075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f0b992eb36b64b69906559fcbcc322b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:33.901825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f0b992eb36b64b69906559fcbcc322b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:33.972390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f0b992eb36b64b69906559fcbcc322b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:33.979738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1113/2266] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:54:33 2022] DELETE /identity/v3/users/f0b992eb36b64b69906559fcbcc322b9 => generated 0 bytes in 408 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:54:33.984495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:54:33.990396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:54:33.990396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:54:34.003097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 13:54:34.004316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:54:34.064578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1154/2267] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 13:54:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 13:54:34.074708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:54:34.076128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:54:34.137798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:54:34.153410 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d792a06e-d4d6-4101-91fe-4cad0d00732d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:54:34.157147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:54:34.157795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:54:34.158073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:54:34.158342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:54:34.168272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:54:34.169788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:54:34.181227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1114/2268] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:54:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:54:34.651900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:54:34.651900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:54:34.666474 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d90412-dc4e-4b6f-9ed1-72e5cb2d46ad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:54:34.672026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:54:34.674435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:54:34.674435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:54:34.674435 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] PATH_INFO: `/v3/projects/7d7f6067714a447c9564063d308e29f6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:54:34.675936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7d7f6067714a447c9564063d308e29f6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:54:34.679737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:54:34.801071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7d7f6067714a447c9564063d308e29f6', 'request_id': 'req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:34.807314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7d7f6067714a447c9564063d308e29f6', 'request_id': 'req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:34.812896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7d7f6067714a447c9564063d308e29f6', 'request_id': 'req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:34.932302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] The token cache is being invalidate because project 7d7f6067714a447c9564063d308e29f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:54:34.932598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:34.933353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:54:34.934935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1155/2269] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:54:34 2022] DELETE /identity/v3/projects/7d7f6067714a447c9564063d308e29f6 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:54:44.227456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:54:44.928700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:55:46.018768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:46.020212 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:46.124537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09165d2-dc9b-40bc-a127-1948450ec7f4 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:46.186691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:46.187374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:46.187665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:46.188027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:46.190047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:46.203700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:46.205360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:46.217451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1115/2270] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:55:46.245242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:46.246815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:46.309803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bbbdd9-d92a-43a8-a215-b9a31ef6b6e3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:46.313536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:46.314196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:46.314478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:46.314754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:46.316611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:46.390328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:46.391800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:46.411696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1156/2271] 10.222.0.70 () {60 vars in 1396 bytes} [Mon May 23 13:55:46 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3240 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:55:50.820013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:50.821569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:50.858027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a486382b-3629-466d-81d6-6d03fa29d6fe service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:50.861787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:50.862441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:50.862730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:50.863041 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:50.864999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:50.910229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:55:50.945901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:50.948280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:50.970240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1116/2272] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:55:51.562628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:51.583520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88052cb9-a25c-4114-8cb7-1a27c244ad50 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:51.587345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:51.589186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:51.589186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:51.589422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:51.591280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:51.686236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:51.687693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:51.698283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:55:51.708873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1157/2273] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:55:53.716308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:53.718208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:53.836515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea139062-21cc-4639-bfe5-c6d9db799579 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:53.841717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:53.843026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:53.843415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:53.843791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:53.846277 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:53.943334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:53.945175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:53.979952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1117/2274] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:55:54.044148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:54.054707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6ec31e-bd9a-43d5-afd7-cef24b758233 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:54.061938 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:54.062624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:54.063860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:54.063860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:54.065719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:54.169495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:54.172764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:54.203457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1158/2275] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:55:54.310732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:54.320034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f69607-284c-4583-956e-a52659ad61c6 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:54.328828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:54.331329 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:54.332061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:54.333587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:54.340274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:54.421599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:55:54.468276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:54.470568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:54.500181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1118/2276] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 190 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:55:56.625622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:55:56.627069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:56.642990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5241bec-0604-4d7e-b6ab-2992af7c4938 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:55:56.648300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:55:56.648995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:55:56.649284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:55:56.649603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:55:56.651693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:55:56.664503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:55:56.666665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:55:56.677118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1159/2277] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:55:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:56:01.383143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:56:01.385634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:01.403624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1670e343-3328-40bb-bcaa-7ccb4ee65ac1 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:56:01.408643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:56:01.409519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:56:01.409929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:56:01.410173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] PATH_INFO: `/v3/users/86ffd5c4601c4ccf90d09be72f41a33f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:56:01.411389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=86ffd5c4601c4ccf90d09be72f41a33f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:56:01.413384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:56:01.657479 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86ffd5c4601c4ccf90d09be72f41a33f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:01.665387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86ffd5c4601c4ccf90d09be72f41a33f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:01.678086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86ffd5c4601c4ccf90d09be72f41a33f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:01.678826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86ffd5c4601c4ccf90d09be72f41a33f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:01.750416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86ffd5c4601c4ccf90d09be72f41a33f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:01.757431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1119/2278] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:56:01 2022] DELETE /identity/v3/users/86ffd5c4601c4ccf90d09be72f41a33f => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:56:02.147440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:56:02.163415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941dd61c-7597-437b-af54-2389add95ff3 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:56:02.224246 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:56:02.225144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:56:02.225423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:56:02.225696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] PATH_INFO: `/v3/projects/8441389e662041a1945ee7164ad17159` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:56:02.226875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8441389e662041a1945ee7164ad17159)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:56:02.228879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:56:02.358495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8441389e662041a1945ee7164ad17159', 'request_id': 'req-9a281481-b82b-4663-aeae-e993e6adc1d1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.365634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8441389e662041a1945ee7164ad17159', 'request_id': 'req-9a281481-b82b-4663-aeae-e993e6adc1d1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.372795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8441389e662041a1945ee7164ad17159', 'request_id': 'req-9a281481-b82b-4663-aeae-e993e6adc1d1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.374771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:56:02.534682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] The token cache is being invalidate because project 8441389e662041a1945ee7164ad17159 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:56:02.535243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.536283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.539033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1160/2279] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:56:02 2022] DELETE /identity/v3/projects/8441389e662041a1945ee7164ad17159 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:56:02.544042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:56:02.545647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:02.559956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a804658-06d4-4bb0-bc3e-fdafcac21954 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:56:02.624483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:56:02.625385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:56:02.625778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:56:02.626128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] PATH_INFO: `/v3/users/e1fe58080a164ff8bd5f46006243a947` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:56:02.640343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fe58080a164ff8bd5f46006243a947)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:56:02.642337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:56:02.734317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:56:02.873271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fe58080a164ff8bd5f46006243a947'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.880709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fe58080a164ff8bd5f46006243a947'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.892528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fe58080a164ff8bd5f46006243a947'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.893251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fe58080a164ff8bd5f46006243a947'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.962242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fe58080a164ff8bd5f46006243a947'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:02.968915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1120/2280] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 13:56:02 2022] DELETE /identity/v3/users/e1fe58080a164ff8bd5f46006243a947 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 13:56:03.289152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:56:03.293611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:03.325985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a281481-b82b-4663-aeae-e993e6adc1d1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:56:03.398314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:56:03.399799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:56:03.400080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:56:03.400351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] PATH_INFO: `/v3/projects/726d8b40614546449ec3c3d3133530c7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:56:03.411002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] RBAC: Authorizing `identity:delete_project(project_id=726d8b40614546449ec3c3d3133530c7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:56:03.413034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:56:03.552858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '726d8b40614546449ec3c3d3133530c7', 'request_id': 'req-e3758d45-3c65-4c3f-9602-629e654c993f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:03.560852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '726d8b40614546449ec3c3d3133530c7', 'request_id': 'req-e3758d45-3c65-4c3f-9602-629e654c993f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:03.566843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '726d8b40614546449ec3c3d3133530c7', 'request_id': 'req-e3758d45-3c65-4c3f-9602-629e654c993f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:03.666610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] The token cache is being invalidate because project 726d8b40614546449ec3c3d3133530c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 13:56:03.667007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:03.667674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 13:56:03.675674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1161/2281] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 13:56:03 2022] DELETE /identity/v3/projects/726d8b40614546449ec3c3d3133530c7 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 13:56:12.979700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=6,keystone:SELECT=19,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:56:13.661629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=57 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:56:52.771228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:56:52.772847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:52.850868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fcf95e-e856-4912-98b9-087743cc366c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:56:52.885285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:56:52.887166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:52.920794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:56:52.921474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:56:52.921763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:56:52.922450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:56:52.923977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:52.976204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3758d45-3c65-4c3f-9602-629e654c993f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:56:52.982666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:56:52.983339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:56:52.983644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:56:52.983915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:56:52.986093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:56:53.001794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:56:53.003425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:56:53.024044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1121/2282] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:56:53.072094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:56:53.074055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:56:53.097750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1162/2283] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:56:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:57:03.021138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:57:03.094749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=64 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:57:46.482643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:57:46.484160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:46.568303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0f5600-e2e3-44c8-bd59-45eb9b505d1c service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:57:46.574959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:57:46.576234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:57:46.576864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:57:46.577422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:57:46.580668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:46.626464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:57:46.626464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:57:46.639919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1122/2284] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:57:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:57:50.748200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:57:50.749869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:50.763833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5279e7e-524f-452f-96fa-0551467ce211 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:57:50.767597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:57:50.768336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:57:50.768709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:57:50.768991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:57:50.771367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:50.782818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:57:50.784642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:57:50.794753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1163/2285] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:57:50 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:57:54.961500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:57:54.963007 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:55.062823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1bee39-65f4-44cc-8c28-c2ca2cbf3ddc service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:57:55.067198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:57:55.073297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:57:55.073297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:57:55.073297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:57:55.073297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:55.171020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:57:55.172747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:57:55.188642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1123/2286] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:57:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 13:57:58.607369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 13:57:58.623684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8c4deb-07d6-42e4-ae29-688ae9f62f38 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 13:57:58.636646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 13:57:58.637257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 13:57:58.637546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 13:57:58.637833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 13:57:58.639960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 13:57:58.657242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 13:57:58.660213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 13:57:58.688017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1164/2287] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 13:57:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 13:58:05.182762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 13:58:08.668444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=12 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:00:50.054473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:00:50.055998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:50.133668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a70705-4b0b-4aef-8a36-e791c6f48531 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:00:50.142059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:00:50.142839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:00:50.143132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:00:50.143409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:00:50.145354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:50.158508 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:00:50.160126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:00:50.171124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1124/2288] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:00:50 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:00:50.464005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:00:50.474216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26032f5-318d-4bd3-8781-8ddbec31ce2c service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:00:50.477940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:00:50.478585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:00:50.478870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:00:50.479144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:00:50.480941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:50.494180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:00:50.495596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:00:50.506457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1165/2289] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:00:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:00:53.714824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:00:53.723567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:53.747434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71478147-64ed-4725-b1ef-26d843283915 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:00:53.754837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:00:53.755654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:00:53.756056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:00:53.756428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:00:53.758701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:53.842865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:00:53.844625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:00:53.854977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1125/2290] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:00:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:00:53.965917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:00:53.978329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc551a29-b72c-47bc-92e4-c690f2572e22 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:00:53.984595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:00:53.985245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:00:53.985529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:00:53.985815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:00:53.987602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:54.061731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:00:54.063343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:00:54.072973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1166/2291] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:00:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:00:54.753739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:00:54.765756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85337769-1f69-4fde-976f-e385dc79d0be service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:00:54.772560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:00:54.773374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:00:54.773660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:00:54.773946 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:00:54.776643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:00:54.862369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:00:54.863998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:00:54.875154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1126/2292] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:00:54 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:01:04.068540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=39 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:01:04.869610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:01:51.708256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:01:51.717674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f29833-c1e4-4b32-8ed6-c333cb3e34c7 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:01:51.722163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:01:51.722819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:51.723108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:51.723384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:51.732529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:01:51.733887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:01:51.743573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1167/2293] 10.222.0.70 () {60 vars in 1396 bytes} [Mon May 23 14:01:51 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:01:52.273134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:01:52.286566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993e3b13-7b14-44b7-8dba-1cce281a2856 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:01:52.292928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dbaba51-20e7-43ea-9421-7b23194729d3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:01:52.293710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaba51-20e7-43ea-9421-7b23194729d3 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:52.294101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaba51-20e7-43ea-9421-7b23194729d3 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:52.294663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dbaba51-20e7-43ea-9421-7b23194729d3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:52.304627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaba51-20e7-43ea-9421-7b23194729d3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:01:52.306519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dbaba51-20e7-43ea-9421-7b23194729d3 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:01:52.317252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1127/2294] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:01:52 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:01:55.446823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:01:55.462171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73017af-7327-4f4b-9ab9-826e1dc29bfd service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:01:55.465784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:01:55.466438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:55.466717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:55.466993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:55.469759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] Could not find project: 8441389e662041a1945ee7164ad17159.: keystone.exception.ProjectNotFound: Could not find project: 8441389e662041a1945ee7164ad17159. May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application Traceback (most recent call last): May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/auth.py", line 285, in get May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:validate_token') May 23 14:01:55.503789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 422, in enforce_call May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application subj_token_target_data = cls._extract_subject_token_target_data() May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 261, in _extract_subject_token_target_data May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application token = PROVIDER_APIS.token_provider_api.validate_token( May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 145, in validate_token May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application token = self._validate_token(token_id) May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 14:01:55.506943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/token/provider.py", line 179, in _validate_token May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application token.mint(token_id, issued_at) May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 579, in mint May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self._validate_token_resources() May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 464, in _validate_token_resources May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application if self.project and not self.project.get('enabled'): May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/models/token_model.py", line 162, in project May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application self.__project = PROVIDERS.resource_api.get_project( May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 23 14:01:55.508702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.get_or_create( May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application with Lock( May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._enter() May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.creator() May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application created_value = creator( May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 23 14:01:55.510192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 23 14:01:55.511877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: 8441389e662041a1945ee7164ad17159. May 23 14:01:55.511877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: ERROR keystone.server.flask.application May 23 14:01:55.511877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1168/2295] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:01:55 2022] GET /identity/v3/auth/tokens => generated 113 bytes in 60 msecs (HTTP/1.1 404) 5 headers in 184 bytes (2 switches on core 0) May 23 14:01:56.965127 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:56.965638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:56.965736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:57.082141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:01:57.083666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:01:57.215553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1128/2296] 10.222.0.70 () {58 vars in 967 bytes} [Mon May 23 14:01:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:01:57.371429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:01:57.373050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:01:57.513103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7524ea54-356f-4664-a67c-9a8ffb5b5d9b service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:01:57.522711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:01:57.524072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:57.524363 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:57.524836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:57.543477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:01:57.545874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:01:57.557115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1169/2297] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 14:01:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 14:01:57.868842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:01:57.892980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d3946a-a83e-4861-835d-74002b0920d3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:01:57.899172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:01:57.901613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:57.901613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:57.901613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:57.902432 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:01:57.916447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:01:57.918372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23adca9-18b3-4fae-bf6f-fa4786a18b3b service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:01:57.940899 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1129/2298] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:01:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 74 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:01:59.099980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:01:59.111337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf729c1c-17bf-42b1-87a5-e4665a1d0eab service swift] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:01:59.118990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:01:59.118990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:01:59.118990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:01:59.118990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:01:59.119680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:01:59.186313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:01:59.187837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:01:59.199277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1170/2299] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:00.703141 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f44b-9487-4702-b0a3-4b5e4878667d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:00.703614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f44b-9487-4702-b0a3-4b5e4878667d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:00.704286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4480f44b-9487-4702-b0a3-4b5e4878667d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:00.748552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4480f44b-9487-4702-b0a3-4b5e4878667d None None] MFA Rules not processed for user `b3eae736c69843ceadc6cbdb258b7195`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:02:00.749852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4480f44b-9487-4702-b0a3-4b5e4878667d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:00.758641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1130/2300] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:02:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 58 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:02:01.107196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:02:01.122026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d5be52-a41a-4a1d-8cbc-aeddf80e6353 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:02:01.126367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:02:01.127053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:01.127339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:01.127604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:01.129589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:01.140230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:02:01.142260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:02:01.151538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1171/2301] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:02:01 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:01.643027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:01.643528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:01.644523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:01.695999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] MFA Rules not processed for user `cf30fa512142489a8dea40ce426a5f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:02:01.697918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:01.713871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1131/2302] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:02:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:02:02.108024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:02:02.120095 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a33068-1e14-4943-9356-097cc08af207 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:02:02.123998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:02:02.124688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:02.125692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:02.126045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:02.127860 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:02.139552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:02:02.142737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:02:02.153031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1172/2303] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:02.670579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:02:02.672414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:02.698467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee5a588-4161-47cc-a168-1508aa87afda None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:02:02.710066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:02:02.712607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:02.712607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:02.712607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:02.715534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:02.733350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:02:02.735868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:02:02.747289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1132/2304] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:02:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:12.146879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:02:12.742487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=79 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:02:18.028996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:02:18.045437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3e4f25-6120-4499-8bff-8776f9d8bba3 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:02:18.063666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:02:18.064347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:18.064664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:18.064949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:18.066775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:02:18.080076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:02:18.081830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:02:18.093782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1173/2305] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:02:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:28.088715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:02:46.532719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:02:46.542686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da90308-44fb-4655-b224-3d68ef86e5b0 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:02:46.546378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:02:46.547074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:46.547360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:46.547640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:46.559961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:02:46.561472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:02:46.574519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1133/2306] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:02:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:55.983676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:02:55.995700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3996fb6e-b757-4554-81b0-e1166190acf6 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:02:56.002063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408b2477-f512-4592-8762-7a2d391f16f3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:02:56.002717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b2477-f512-4592-8762-7a2d391f16f3 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:02:56.003004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b2477-f512-4592-8762-7a2d391f16f3 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:02:56.003284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408b2477-f512-4592-8762-7a2d391f16f3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:02:56.013220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b2477-f512-4592-8762-7a2d391f16f3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:02:56.014725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408b2477-f512-4592-8762-7a2d391f16f3 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:02:56.043013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1174/2307] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:02:55 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:02:56.569234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:03:06.040151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=13 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:03:52.713904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:03:52.722985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e81766-983d-4c07-a2ae-1d669ef791f3 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:03:52.727617 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:03:52.728291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:03:52.728597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:03:52.728897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:03:52.737339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:03:52.738873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:03:52.748496 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1134/2308] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:03:52 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:04:00.078832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:04:00.079538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:04:00.079538 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:04:00.115359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] MFA Rules not processed for user `a58edb91d7b641298e40dd356eec9eac`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:04:00.117334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:04:00.124475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1175/2309] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:04:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:04:02.744353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:04:10.111871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:05:50.762102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:05:50.783971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2533b4e0-670e-466c-b1f9-d0232074af6e service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:05:50.788467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:05:50.789169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:05:50.789468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:05:50.790156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:05:50.826150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:05:50.827607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:05:50.839846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1135/2310] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:05:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:05:53.031400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:05:53.078169 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dacae07-645b-49ee-8bdd-3a86a912652b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:05:53.083199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:05:53.083841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:05:53.084124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:05:53.084399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:05:53.093223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:05:53.094796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:05:53.105476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1176/2311] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:05:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:05:56.217004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:05:56.241859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3d69b2-602a-4039-95d3-f135e94021e3 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:05:56.245763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:05:56.246454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:05:56.246734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:05:56.247006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:05:56.257861 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:05:56.259352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:05:56.293636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1136/2312] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:05:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:06:03.100462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=21 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:06:06.289620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=37 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:06:32.043578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:06:32.078461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51440f-f967-4e25-96bd-8283a8fc0cb7 service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:06:32.083833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:06:32.084475 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:06:32.084778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:06:32.085050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:06:32.093663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:06:32.095233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:06:32.122301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1177/2313] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:06:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:06:36.111503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:06:36.111503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:06:36.160211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55833805-91fe-428a-a431-1cd43b4e0414 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:06:36.170924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:06:36.170924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:06:36.170924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:06:36.170924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:06:36.177117 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:06:36.178848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:06:36.213343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1137/2314] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:06:36 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:06:42.119535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=20 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:06:46.209664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:07:47.462632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:07:47.464315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:07:47.546158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab32309c-fa42-45cb-9141-d5c16ecaa7b5 service nova] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:07:47.550872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:07:47.551522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:07:47.551809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:07:47.552089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:07:47.553989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:07:47.640766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:07:47.642224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:07:47.655133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1178/2315] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:07:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:07:56.961076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:07:56.974669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad241bc-e38f-4772-a131-6dfd878d63b0 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:07:56.980362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:07:56.981053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:07:56.981341 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:07:56.981620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:07:56.983548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:07:57.073228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:07:57.074840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:07:57.084657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1138/2316] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:07:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:07:57.648895 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=68 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:07.079926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=33 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:32.572987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:32.639769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f9403f-01d0-48c1-bb98-92cb76a341d8 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:32.644833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:32.645558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:32.645848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:32.646126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:32.648078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:32.728737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:32.731527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:32.749580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1179/2317] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:32 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:35.330096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:35.337015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13df7f2c-97d7-4f54-9be4-d4b37391e7a0 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:35.340769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:35.341706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:35.342054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:35.342413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:35.345304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:35.432994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:35.434512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:35.455851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1139/2318] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:36.124981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:36.126699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:36.138354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6374f7c4-8ca5-4bcb-8fd0-bc7dd2140639 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:36.142703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:36.143354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:36.143640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:36.143913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:36.145694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:36.189725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:36.191188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:36.221290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1180/2319] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:36.502663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:36.518991 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e658d11f-8fe6-4714-8cff-4a5c72683f12 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:36.527443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:36.528466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:36.529084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:36.529557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:36.547428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:36.550696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:36.573802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1140/2320] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:37.818384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:37.833064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30634559-8a1b-465a-9d02-f640028eb90c service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:37.837433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:37.838111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:37.838396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:37.838672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:37.841379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:37.865173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:37.867060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a5f383-17f8-430e-82e0-c6735f6f67e8 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:37.876421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:37.884160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1181/2321] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:37.910545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-160e32c9-ce43-4d11-91e1-47539a9fb791 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:37.930289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83fd80c-3f08-4329-bc68-f531da345937 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:37.931380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fd80c-3f08-4329-bc68-f531da345937 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:37.931837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fd80c-3f08-4329-bc68-f531da345937 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:37.932196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83fd80c-3f08-4329-bc68-f531da345937 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:37.946914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fd80c-3f08-4329-bc68-f531da345937 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:37.949842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83fd80c-3f08-4329-bc68-f531da345937 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:37.977453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1141/2322] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:38.128362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.128709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.129038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.239804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:08:38.241070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:38.252855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1182/2323] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:08:38.261484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.261694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.262116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.308018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:08:38.318286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:38.334207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1142/2324] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:08:38.338197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:38.344896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:38.350236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc0c14-7050-4916-b152-8235b456c418 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:38.357625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:38.358609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.358896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.359175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.361109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:08:38.361412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:38.363118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:38.377389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1183/2325] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:08:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:08:38.382016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:38.397078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89efc39-84f1-4422-bd79-642e8b30f1d6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:38.402574 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:38.403259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.403544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.403825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.404534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:38.407683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:38.596882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1143/2326] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:08:38 2022] POST /identity/v3/projects => generated 391 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:08:38.598460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:38.612844 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468f3b8a-f401-4a00-aeef-dfbb0b5d461e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:38.749182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:38.752603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.752603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.752603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.754980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:38.762008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:38.882747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1184/2327] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:08:38 2022] POST /identity/v3/users => generated 296 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:08:38.886446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:38.904316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fe19e9-2c80-4a50-97b8-b09d4804ac39 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:38.912294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:38.914637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.915008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.916532 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.917402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:38.920191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:38.947023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1144/2328] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:08:38.959481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:38.977205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95c073aa-6a47-4070-85de-166550f72c7a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:38.983456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb881bc-48d1-4bf6-a3aa-83d3e88b9259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:38.985322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb881bc-48d1-4bf6-a3aa-83d3e88b9259 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:38.987337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb881bc-48d1-4bf6-a3aa-83d3e88b9259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:38.987771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb881bc-48d1-4bf6-a3aa-83d3e88b9259 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:38.989143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb881bc-48d1-4bf6-a3aa-83d3e88b9259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:38.992425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb881bc-48d1-4bf6-a3aa-83d3e88b9259 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:39.010428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1185/2329] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:08:38 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:08:39.014288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:39.026263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5d6a04-a37d-4beb-b0b3-e6fa673c5aed admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:39.032774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:39.034273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:39.034731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:39.035176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] PATH_INFO: `/v3/projects/a114e06453194f52a3ccdfe6e332f426/users/f3d5d4ed6758408cb0c65c43bbd3fff0/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:39.056785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a114e06453194f52a3ccdfe6e332f426, user_id=f3d5d4ed6758408cb0c65c43bbd3fff0, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:39.059708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:39.153282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1145/2330] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:08:39 2022] PUT /identity/v3/projects/a114e06453194f52a3ccdfe6e332f426/users/f3d5d4ed6758408cb0c65c43bbd3fff0/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:08:39.172649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:39.173178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:39.173715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:39.227989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] MFA Rules not processed for user `f3d5d4ed6758408cb0c65c43bbd3fff0`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:08:39.230483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:39.336873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:39.341325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:39.383766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1186/2331] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:08:39 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:08:39.389681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:39.397904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:39.399522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:39.451602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b429e5-cabf-41fb-8e0d-3013fca19d31 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:39.461457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:39.462144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:39.462437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:39.462847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:39.466523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:39.483427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:39.486111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:39.501477 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1146/2332] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:39.515602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:39.517795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bdeb-6628-40fc-916c-e025d83ba227 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:39.523609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:39.524389 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:39.524869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:39.525182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:39.538180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:39.538180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:39.554137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1187/2333] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:39 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:08:41.323557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:41.432368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cda8ad-65a2-4f88-8176-f5b3085e4cb4 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:41.442496 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:41.443285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:41.443524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:41.444094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:41.446124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:41.475715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:41.477421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:41.488659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1147/2334] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:41.558541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:41.560032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:41.641312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7e1f04-40f9-4e5a-8f01-cd4bc6e4ec6d service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:41.647337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:41.648477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:41.648967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:41.649334 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] PATH_INFO: `/v3/users/a58edb91d7b641298e40dd356eec9eac` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:41.650790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a58edb91d7b641298e40dd356eec9eac)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:41.654893 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:41.846579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a58edb91d7b641298e40dd356eec9eac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:41.847361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a58edb91d7b641298e40dd356eec9eac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:41.847961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a58edb91d7b641298e40dd356eec9eac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:41.860339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a58edb91d7b641298e40dd356eec9eac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:41.867264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6ab4a22b-bec0-44ed-876f-058cd8b46b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a58edb91d7b641298e40dd356eec9eac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:41.945632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1188/2335] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:08:41 2022] DELETE /identity/v3/users/a58edb91d7b641298e40dd356eec9eac => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:08:42.200825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:42.211545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66eedd2-43f3-450c-9c73-07bb182ebbfb service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:42.308492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:42.309446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:42.309724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:42.309996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] PATH_INFO: `/v3/projects/6bb9e74e9b64480a80d45a03945ea51f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:42.311293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bb9e74e9b64480a80d45a03945ea51f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:42.313257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:42.484893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:42.509350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bb9e74e9b64480a80d45a03945ea51f', 'request_id': 'req-9c0917e3-002b-4d84-86ae-f7cb8c841f28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:42.517124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bb9e74e9b64480a80d45a03945ea51f', 'request_id': 'req-9c0917e3-002b-4d84-86ae-f7cb8c841f28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:42.524452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bb9e74e9b64480a80d45a03945ea51f', 'request_id': 'req-9c0917e3-002b-4d84-86ae-f7cb8c841f28'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:42.640137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] The token cache is being invalidate because project 6bb9e74e9b64480a80d45a03945ea51f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:08:42.640607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:42.641371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9c0917e3-002b-4d84-86ae-f7cb8c841f28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:08:42.643572 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1148/2336] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:08:42 2022] DELETE /identity/v3/projects/6bb9e74e9b64480a80d45a03945ea51f => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:08:42.648967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:42.649443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:42.649536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:42.690452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:08:42.691305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:42.786850 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1189/2337] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:08:42.794902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:42.795188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:42.795507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:42.815869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:08:42.817134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:42.825879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1149/2338] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:08:42.830365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:42.840939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8277c3-d451-46e6-9e8f-bd20c0e6e4cc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:42.846030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:42.846798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:42.847089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:42.847369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:42.848186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:08:42.848479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:42.850137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:42.857698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:42.863703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1190/2339] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:08:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:08:42.868327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:42.878615 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a98390-04f5-4ad1-ba77-829b95d5d4fd None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:42.883667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:42.884227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:42.884507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:42.884807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:42.885491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:42.887419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:42.991724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1150/2340] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:08:42 2022] POST /identity/v3/projects => generated 383 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:08:42.991845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:43.002019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba08f55-0b22-4bd5-88bb-006898f042e1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:43.062816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:43.063488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:43.063779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:43.064062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:43.064782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:43.066705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:43.137649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1191/2341] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:08:42 2022] POST /identity/v3/users => generated 292 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:08:43.142729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:43.153171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d01db6-c021-4ec2-b1f0-efe48c38280f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:43.158466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:43.159235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:43.159527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:43.159806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:43.160544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:43.162196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:43.173313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1151/2342] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:08:43.178367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:43.188811 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6c7087-55db-488d-8fd6-59ed53f2fd77 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:43.194321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e053fc9e-e5fc-4206-80ee-03099550bb99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:43.195114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053fc9e-e5fc-4206-80ee-03099550bb99 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:43.195425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053fc9e-e5fc-4206-80ee-03099550bb99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:43.195699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e053fc9e-e5fc-4206-80ee-03099550bb99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:43.196426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053fc9e-e5fc-4206-80ee-03099550bb99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:43.198050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e053fc9e-e5fc-4206-80ee-03099550bb99 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:43.209106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1192/2343] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:08:43 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:08:43.213366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:43.224210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c07a8e-5f46-4e51-8616-17e3970e6064 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:43.228817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:43.229693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:43.229973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:43.230247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] PATH_INFO: `/v3/projects/9598a5227dfe4afaa88ba746097f9692/users/51ad4e3e2edb41c5bee9bd61025c6cdc/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:43.245407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9598a5227dfe4afaa88ba746097f9692, user_id=51ad4e3e2edb41c5bee9bd61025c6cdc, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:43.248255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:43.328424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1152/2344] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:08:43 2022] PUT /identity/v3/projects/9598a5227dfe4afaa88ba746097f9692/users/51ad4e3e2edb41c5bee9bd61025c6cdc/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:08:43.338970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:43.340612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:43.340612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:43.366610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] MFA Rules not processed for user `51ad4e3e2edb41c5bee9bd61025c6cdc`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:08:43.367837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:43.435055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1193/2345] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:08:43 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:08:43.446620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:08:43.448332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:08:43.521172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439ca59f-5ef3-4780-9b38-1f3f81074070 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:08:43.526636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:08:43.527280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:08:43.527562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:08:43.527837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:08:43.537022 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:08:43.538633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:08:43.547565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1153/2346] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:08:43 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:08:53.432181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=67,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:08:53.543538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=72,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:03.424618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:03.489069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875abea7-5dbd-4515-883e-9e7edd4aa62b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:03.494037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:03.494935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:03.495224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:03.495505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] PATH_INFO: `/v3/users/51ad4e3e2edb41c5bee9bd61025c6cdc` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:03.496778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=51ad4e3e2edb41c5bee9bd61025c6cdc)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:03.498735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:03.722673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51ad4e3e2edb41c5bee9bd61025c6cdc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:03.723465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51ad4e3e2edb41c5bee9bd61025c6cdc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:03.724073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51ad4e3e2edb41c5bee9bd61025c6cdc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:03.741369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51ad4e3e2edb41c5bee9bd61025c6cdc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:03.754666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51ad4e3e2edb41c5bee9bd61025c6cdc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:03.774873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:03.777210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:03.852242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267683f0-2edf-4086-8ad0-f2a89bcb92bc service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:03.855737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:03.856396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:03.856786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:03.857389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:03.864823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:03.871553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:03.873215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05f876a3-deff-45fd-912a-6b7619ca4ad2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:03.882267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1154/2347] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:03.900123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1194/2348] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:03 2022] DELETE /identity/v3/users/51ad4e3e2edb41c5bee9bd61025c6cdc => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:03.913103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:03.913798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:03.913798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:03.946133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:03.947037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:04.035290 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1155/2349] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:04.047044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:04.049978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:04.138996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee21b36-e54e-4d5d-830d-710ee94b9f71 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:04.147295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde6c77-2dd8-4a7a-a791-34abaac3ef53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:04.148756 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6c77-2dd8-4a7a-a791-34abaac3ef53 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:04.149350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6c77-2dd8-4a7a-a791-34abaac3ef53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:04.149865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde6c77-2dd8-4a7a-a791-34abaac3ef53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:04.167900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6c77-2dd8-4a7a-a791-34abaac3ef53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:04.172103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde6c77-2dd8-4a7a-a791-34abaac3ef53 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:04.204050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1195/2350] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:05.236406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:05.241247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:05.256713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1c93d7-8cbb-445d-b8d2-3ca20945f00d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:05.263476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:05.268968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:05.271337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:05.271694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] PATH_INFO: `/v3/projects/9598a5227dfe4afaa88ba746097f9692` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:05.273371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] RBAC: Authorizing `identity:delete_project(project_id=9598a5227dfe4afaa88ba746097f9692)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:05.280333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:05.446568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9598a5227dfe4afaa88ba746097f9692', 'request_id': 'req-2fc959f1-e27e-4d12-8c31-40194bfe7add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:05.455243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9598a5227dfe4afaa88ba746097f9692', 'request_id': 'req-2fc959f1-e27e-4d12-8c31-40194bfe7add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:05.465921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9598a5227dfe4afaa88ba746097f9692', 'request_id': 'req-2fc959f1-e27e-4d12-8c31-40194bfe7add'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:05.623119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] The token cache is being invalidate because project 9598a5227dfe4afaa88ba746097f9692 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:05.623546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:05.629734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2fc959f1-e27e-4d12-8c31-40194bfe7add admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:05.635519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1156/2351] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:05 2022] DELETE /identity/v3/projects/9598a5227dfe4afaa88ba746097f9692 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:05.639685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acee29d-0519-46f1-8237-9176f620817a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:05.640678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acee29d-0519-46f1-8237-9176f620817a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:05.641214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acee29d-0519-46f1-8237-9176f620817a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:05.664079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9acee29d-0519-46f1-8237-9176f620817a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:05.665320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9acee29d-0519-46f1-8237-9176f620817a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:05.687416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:05.740447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1196/2352] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:05.748943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:05.749230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:05.749511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:05.771853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:05.779994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:05.782916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1157/2353] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:05.793377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acee29d-0519-46f1-8237-9176f620817a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:05.804623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acee29d-0519-46f1-8237-9176f620817a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:05.810107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:05.811098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:05.811098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:05.811098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:05.814941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:09:05.814941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:05.814941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:05.831473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1197/2354] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:09:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:09:05.834008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:05.846308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c668e23f-10f7-48c5-8610-09ed6abb8b56 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:05.852813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:05.853555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:05.853845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:05.854125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:05.854846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:05.856961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:05.960683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:05.971759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1158/2355] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:05 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:05.977402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:05.995342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710ec8b5-9a31-4eb5-840a-f183063a3af3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:06.063738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:06.064430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.064759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.065043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.065751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:06.068482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:06.188508 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1198/2356] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:05 2022] POST /identity/v3/users => generated 283 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:06.189837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:06.201921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321962a2-a21a-4204-add1-4062222188e5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:06.207726 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:06.209040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.209788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.210298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.211311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:06.214179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:06.227649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1159/2357] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:06.231702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:06.244356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdecc737-0b6c-4cb1-9d60-f762750276c6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:06.261116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec601b9-32ba-41fb-8717-18a0b2e69160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:06.261914 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec601b9-32ba-41fb-8717-18a0b2e69160 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.262283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec601b9-32ba-41fb-8717-18a0b2e69160 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.262619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec601b9-32ba-41fb-8717-18a0b2e69160 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.263459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec601b9-32ba-41fb-8717-18a0b2e69160 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:06.265258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec601b9-32ba-41fb-8717-18a0b2e69160 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:06.281743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1199/2358] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:06 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:06.283017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:06.295377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ec1499-8b43-4957-9540-e3d951e9e5c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:06.302696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:06.305876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.305876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.305876 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] PATH_INFO: `/v3/projects/ec6737030ec04259a0a68dee77aea5f7/users/baae8e60403746d08c8f12394b481e0a/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.322138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec6737030ec04259a0a68dee77aea5f7, user_id=baae8e60403746d08c8f12394b481e0a, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:06.325273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:06.509957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1160/2359] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:06 2022] PUT /identity/v3/projects/ec6737030ec04259a0a68dee77aea5f7/users/baae8e60403746d08c8f12394b481e0a/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:06.522606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.522903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.523278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.528261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:06.530913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:06.563638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] MFA Rules not processed for user `baae8e60403746d08c8f12394b481e0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:06.565028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:06.626500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f327376d-6da9-4277-8fb3-71e3473a2cdd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:06.633143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:06.635347 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.635629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.635910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] PATH_INFO: `/v3/users/f3d5d4ed6758408cb0c65c43bbd3fff0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.637281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3d5d4ed6758408cb0c65c43bbd3fff0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:06.639564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:06.727065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1200/2360] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:06.734002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:06.763357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1428a8b9-380f-4706-a2b2-417c17ed0167 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:06.771523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b9f546-4f4d-4489-a6bc-d45635c199ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:06.772945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9f546-4f4d-4489-a6bc-d45635c199ee admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.773555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9f546-4f4d-4489-a6bc-d45635c199ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.774297 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b9f546-4f4d-4489-a6bc-d45635c199ee admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.775688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9f546-4f4d-4489-a6bc-d45635c199ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:06.779968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b9f546-4f4d-4489-a6bc-d45635c199ee admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:06.920167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1201/2361] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:06 2022] POST /identity/v3/projects => generated 365 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:09:06.923056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:06.923615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:06.923790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3d5d4ed6758408cb0c65c43bbd3fff0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:06.924271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:06.935414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3d5d4ed6758408cb0c65c43bbd3fff0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:06.945391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:06.946746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:06.956504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3d5d4ed6758408cb0c65c43bbd3fff0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:06.957535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3d5d4ed6758408cb0c65c43bbd3fff0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:07.018222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1202/2362] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 14:09:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:07.018869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.055166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-9d9d94d4-36e0-455f-9c02-da9624db238e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3d5d4ed6758408cb0c65c43bbd3fff0'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:07.065218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1161/2363] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:06 2022] DELETE /identity/v3/users/f3d5d4ed6758408cb0c65c43bbd3fff0 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:07.067534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679be53f-2daf-4860-b791-946df16df500 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.067839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679be53f-2daf-4860-b791-946df16df500 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.068122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679be53f-2daf-4860-b791-946df16df500 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.092811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-679be53f-2daf-4860-b791-946df16df500 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:07.095931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-679be53f-2daf-4860-b791-946df16df500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:07.099082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:07.131376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9b15bb5-0292-43ea-b802-91b59d88cfea None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.212716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1162/2364] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:07.226906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679be53f-2daf-4860-b791-946df16df500 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.228768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-679be53f-2daf-4860-b791-946df16df500 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:07.230378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.231346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.231848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.232135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.233691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.235352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.273580 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:07.305474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-679be53f-2daf-4860-b791-946df16df500 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.309446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.313754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.313754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.313754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.321933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.323904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.336033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1163/2365] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:07.337361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.338912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:07.382707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1203/2366] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:07 2022] POST /identity/v3/users => generated 289 bytes in 364 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:07.388515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.397533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:07.420104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ed8070-95ad-4914-b94f-31b7c952e8a0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.426461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.427777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.427777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.428405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.429234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.431133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.445605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1204/2367] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:09:07.451339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.464798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-490bfa00-4eb4-4a5f-a49a-43a5affc10bf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.470624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.473487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.473487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.473487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] PATH_INFO: `/v3/projects/1c0591459edb46abbcba537495c39a83/users/d46d97a40cf84f1a873b287e71ae51a4/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.496395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0591459edb46abbcba537495c39a83, user_id=d46d97a40cf84f1a873b287e71ae51a4, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.499546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.512476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac1acf7-6ffb-47e3-a479-5b97351a9bf8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.522414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.523552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.523996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.524227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.527122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:07.611560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1205/2368] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:07 2022] PUT /identity/v3/projects/1c0591459edb46abbcba537495c39a83/users/d46d97a40cf84f1a873b287e71ae51a4/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:07.614725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.616137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.617869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.634247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1164/2369] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 14:09:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 14:09:07.731276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91433beb-959f-4d8b-b94c-8d7d70957faf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.737488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ce172-4ee1-4624-a198-e23f1a5b0e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.738206 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce172-4ee1-4624-a198-e23f1a5b0e5e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.738879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce172-4ee1-4624-a198-e23f1a5b0e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.739028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ce172-4ee1-4624-a198-e23f1a5b0e5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.739951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ce172-4ee1-4624-a198-e23f1a5b0e5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.741739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ce172-4ee1-4624-a198-e23f1a5b0e5e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.755768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1206/2370] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:07 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:09:07.760043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:07.761933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:07.781325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6bb89e-6d26-45de-bc59-0ac4b8f33205 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:07.793884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:07.795063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.795446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.795923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] PATH_INFO: `/v3/projects/1c0591459edb46abbcba537495c39a83/users/d46d97a40cf84f1a873b287e71ae51a4/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.808669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c0591459edb46abbcba537495c39a83, user_id=d46d97a40cf84f1a873b287e71ae51a4, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:07.808669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:07.845911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1165/2371] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:07 2022] PUT /identity/v3/projects/1c0591459edb46abbcba537495c39a83/users/d46d97a40cf84f1a873b287e71ae51a4/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:07.846739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:07.847424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:07.848284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:07.918196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] MFA Rules not processed for user `d46d97a40cf84f1a873b287e71ae51a4`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:07.919921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:08.042875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1207/2372] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:07 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:08.063360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:08.066386 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:08.109063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:08.169770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:08.174880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:08.255361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353597d1-4b7e-414f-b7e2-d4f623facaad admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:08.264074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:08.265781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:08.266175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:08.267690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:08.297219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:08.298808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:08.299811 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b0a9f-29a4-4322-9a89-006636a8aedc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:08.306170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:08.307046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:08.307345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:08.307732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] PATH_INFO: `/v3/projects/a114e06453194f52a3ccdfe6e332f426` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:08.309000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a114e06453194f52a3ccdfe6e332f426)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:08.310931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:08.314569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1166/2373] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:08 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 252 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:08.502305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:08.525073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a114e06453194f52a3ccdfe6e332f426', 'request_id': 'req-9bcbf39a-acce-472f-8e38-b93e329afcb5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:08.533391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a114e06453194f52a3ccdfe6e332f426', 'request_id': 'req-9bcbf39a-acce-472f-8e38-b93e329afcb5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:08.543243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a114e06453194f52a3ccdfe6e332f426', 'request_id': 'req-9bcbf39a-acce-472f-8e38-b93e329afcb5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:08.609387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:08.627533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bcabc4-4c36-4ddb-9b22-376881bda9e0 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:08.645312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] The token cache is being invalidate because project a114e06453194f52a3ccdfe6e332f426 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:08.645312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:08.647126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9bcbf39a-acce-472f-8e38-b93e329afcb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:08.653666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1208/2374] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:08 2022] DELETE /identity/v3/projects/a114e06453194f52a3ccdfe6e332f426 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:08.711488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:08.712307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:08.712630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:08.712917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:08.715295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:08.841111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:08.842913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:08.856837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1167/2375] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:08 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 248 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:11.406392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:11.407006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:11.407006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:11.429067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] MFA Rules not processed for user `cf30fa512142489a8dea40ce426a5f36`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:11.430413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:11.517789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1209/2376] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:11.920152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:11.924739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:12.036456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb673ca-da77-4fac-a443-9471307b2929 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:12.040867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:12.041673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:12.042062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:12.042441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:12.044476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:12.056282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:12.057963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:12.071972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1168/2377] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:12.613586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:12.615625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:12.733788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db446d1-d72f-46ce-9894-90601cf2e3c9 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:12.740943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:12.741785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:12.742176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:12.742557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:12.744828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:12.758803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:12.760887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6221d8c-f78a-473c-9965-79bf5d66a6ae service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:12.774787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1210/2378] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:22.064378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:22.768371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=6,keystone:SELECT=71,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:26.161071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:26.162561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:26.240512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d1bbc2-8dc0-45b4-8cfe-a4fa7a6cc3f8 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:26.242202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:26.243214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:26.243666 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:26.244238 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] PATH_INFO: `/v3/users/baae8e60403746d08c8f12394b481e0a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:26.245449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] RBAC: Authorizing `identity:delete_user(user_id=baae8e60403746d08c8f12394b481e0a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:26.248833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:26.439266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'baae8e60403746d08c8f12394b481e0a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:26.448556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'baae8e60403746d08c8f12394b481e0a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:26.467131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'baae8e60403746d08c8f12394b481e0a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:26.467866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'baae8e60403746d08c8f12394b481e0a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:26.553059 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'baae8e60403746d08c8f12394b481e0a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:26.559794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1169/2379] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:26 2022] DELETE /identity/v3/users/baae8e60403746d08c8f12394b481e0a => generated 0 bytes in 399 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:26.564318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:26.564869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:26.564971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:26.582793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:26.584006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:26.654101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1211/2380] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:26.683585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:26.685095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:26.790097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afca2ecc-1ee4-4013-9acc-c71f0b2829bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:26.801027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:26.801830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:26.804128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:26.805533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:26.830791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:26.833519 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:26.846823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1170/2381] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:27.327723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:27.329407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:27.342954 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b96ab98-7b93-4293-9e19-e05092e2f3b4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:27.347744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:27.348669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:27.348959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:27.349239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] PATH_INFO: `/v3/projects/ec6737030ec04259a0a68dee77aea5f7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:27.350470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec6737030ec04259a0a68dee77aea5f7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:27.352659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:27.451568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec6737030ec04259a0a68dee77aea5f7', 'request_id': 'req-0b965444-64cf-4993-bd74-b357fba0b08d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.458894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec6737030ec04259a0a68dee77aea5f7', 'request_id': 'req-0b965444-64cf-4993-bd74-b357fba0b08d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.465013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec6737030ec04259a0a68dee77aea5f7', 'request_id': 'req-0b965444-64cf-4993-bd74-b357fba0b08d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.530037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] The token cache is being invalidate because project ec6737030ec04259a0a68dee77aea5f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:27.530439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.531098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.533002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1212/2382] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:27 2022] DELETE /identity/v3/projects/ec6737030ec04259a0a68dee77aea5f7 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:27.536765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:27.546888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c179b9-fc74-4b57-b976-8779c395ba25 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:27.567928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:27.610135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:27.610135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:27.610135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:27.610594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] PATH_INFO: `/v3/users/d46d97a40cf84f1a873b287e71ae51a4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:27.611630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d46d97a40cf84f1a873b287e71ae51a4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:27.614260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:27.784004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd46d97a40cf84f1a873b287e71ae51a4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.791939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd46d97a40cf84f1a873b287e71ae51a4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.804829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd46d97a40cf84f1a873b287e71ae51a4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.805634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd46d97a40cf84f1a873b287e71ae51a4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.876509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-724b1f23-23ce-45cd-b60f-24e75836c719 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd46d97a40cf84f1a873b287e71ae51a4'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:27.883788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1171/2383] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:27 2022] DELETE /identity/v3/users/d46d97a40cf84f1a873b287e71ae51a4 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:28.357847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:28.360650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:28.373554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b965444-64cf-4993-bd74-b357fba0b08d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:28.438836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:28.439713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:28.440000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:28.440276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] PATH_INFO: `/v3/projects/1c0591459edb46abbcba537495c39a83` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:28.441613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c0591459edb46abbcba537495c39a83)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:28.443934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:28.531377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:28.549795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c0591459edb46abbcba537495c39a83', 'request_id': 'req-0515ae1b-4625-48f9-afe4-4e9484b83623'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:28.557491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c0591459edb46abbcba537495c39a83', 'request_id': 'req-0515ae1b-4625-48f9-afe4-4e9484b83623'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:28.563251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c0591459edb46abbcba537495c39a83', 'request_id': 'req-0515ae1b-4625-48f9-afe4-4e9484b83623'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:28.657380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] The token cache is being invalidate because project 1c0591459edb46abbcba537495c39a83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:28.657380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:28.657380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0515ae1b-4625-48f9-afe4-4e9484b83623 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:28.657380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1213/2384] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:28 2022] DELETE /identity/v3/projects/1c0591459edb46abbcba537495c39a83 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:28.662596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:28.663071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:28.663215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:28.682531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:28.683757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:28.747725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1172/2385] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:28.756682 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38441d-c053-47df-abec-556f42e55224 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:28.757235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38441d-c053-47df-abec-556f42e55224 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:28.757704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd38441d-c053-47df-abec-556f42e55224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:28.777737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bd38441d-c053-47df-abec-556f42e55224 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:28.779188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bd38441d-c053-47df-abec-556f42e55224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:28.787831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1214/2386] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:28.792488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:28.802882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e712cb-9a69-4aea-a24f-cd41f965a36e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:28.807810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:28.808485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:28.808816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:28.809244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:28.809992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:09:28.810303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:28.811937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:28.829269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1173/2387] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:09:28.830592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38441d-c053-47df-abec-556f42e55224 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:28.841103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd38441d-c053-47df-abec-556f42e55224 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:28.845815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:28.846482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:28.846770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:28.847052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:28.847947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:28.849936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:28.994388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1215/2388] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:28 2022] POST /identity/v3/projects => generated 369 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:28.995716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.007172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39544ff-19bd-4e02-9766-4d1c9eb1524c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.038052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:29.074280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.074984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.075264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.075541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.076717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.078673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:29.248688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1174/2389] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:28 2022] POST /identity/v3/users => generated 285 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:09:29.252616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.283818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301dac71-28fd-4d8f-a747-4f084d80412d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.291202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.291663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.291953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.292333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.293013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.295487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:29.324096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1216/2390] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:29.329086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.347074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c41935a-0860-4ced-bcac-f862c229268a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.357568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.358177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.358499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.358745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.359500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.360120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.361179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:29.362311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:29.373601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1175/2391] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:29 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:29.377628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.394090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8ff377b-cb6b-4bd1-ac45-6520f44af14c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.406947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8b4e3ef-8e7f-4ee4-8bc3-cdd25a7fba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.407944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4e3ef-8e7f-4ee4-8bc3-cdd25a7fba39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.408231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4e3ef-8e7f-4ee4-8bc3-cdd25a7fba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.408512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b4e3ef-8e7f-4ee4-8bc3-cdd25a7fba39 admin admin] PATH_INFO: `/v3/projects/3b9c25999dd3466784c2bd657cd5a147/users/60f19f40bf324b52b516913de26217b9/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.429694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4e3ef-8e7f-4ee4-8bc3-cdd25a7fba39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9c25999dd3466784c2bd657cd5a147, user_id=60f19f40bf324b52b516913de26217b9, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.432470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8b4e3ef-8e7f-4ee4-8bc3-cdd25a7fba39 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:29.521621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572236a2-c5b9-44c5-a6fd-699d5dfc9af1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.526408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1176/2392] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:29 2022] PUT /identity/v3/projects/3b9c25999dd3466784c2bd657cd5a147/users/60f19f40bf324b52b516913de26217b9/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:09:29.544822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.545766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.545766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.599136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '99a35674d90447d0a7ee255845280be4', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.600036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.600195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] MFA Rules not processed for user `60f19f40bf324b52b516913de26217b9`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:29.600493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.600813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.602199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:29.602806 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:29.630370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:29.678183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.679537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:29.682638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1177/2393] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:29 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:29.687375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.691941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1217/2394] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:29.761462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1896b551-8d62-4ad2-ab33-39aa13060e33 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.766305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.767030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.767314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.767592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.768331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.770848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:29.861683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:29.869252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1178/2395] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:29 2022] POST /identity/v3/projects => generated 367 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:29.874346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.883836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfb6a41-38cc-4769-a387-31260e02a267 service glance] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:29.935247 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:29.937393 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:29.945683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:29.946508 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:29.946885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:29.947133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:29.947846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:29.950014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.000164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49136b2-9130-4d30-a9c3-9863c0bb73ed admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.003802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.004469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.004789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.005061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.006851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:30.016690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.018123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.026988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1179/2396] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:29 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:30.080330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1218/2397] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:29 2022] POST /identity/v3/users => generated 290 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:30.084903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.097122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca0c4fb3-9071-4ab2-a3b9-9edd86050679 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.103265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.103627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.103841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.104144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.104859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.106577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.115990 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.117878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:30.118394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1180/2398] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:30.122720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.131473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773585f5-4cd4-43c0-bbe0-8546b7724213 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.134780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879363a9-c6f4-4b41-9ef5-aba5cc09de3f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.135696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.138684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.138684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.138684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.138684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:30.139490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.140379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.140697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.140980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] PATH_INFO: `/v3/projects/c2fdbb262d8f40099c1cd17f6c8bf42c/users/081c1c018e624104b8a62bd61682f5f3/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.159863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2fdbb262d8f40099c1cd17f6c8bf42c, user_id=081c1c018e624104b8a62bd61682f5f3, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.163335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.222985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.224612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.237161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1219/2399] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:09:30.261020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1181/2400] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:30 2022] PUT /identity/v3/projects/c2fdbb262d8f40099c1cd17f6c8bf42c/users/081c1c018e624104b8a62bd61682f5f3/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:30.265497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.277094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5482121-a13f-4f5f-964f-754d8e9d2dd7 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.313651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:30.353046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f5e80c-19f6-4923-aebe-670ea935356c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.353882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5e80c-19f6-4923-aebe-670ea935356c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.354171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5e80c-19f6-4923-aebe-670ea935356c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.354446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f5e80c-19f6-4923-aebe-670ea935356c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.355181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5e80c-19f6-4923-aebe-670ea935356c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.356816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f5e80c-19f6-4923-aebe-670ea935356c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.371730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1220/2401] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:30 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:30.372620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.386120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86c7d4b-09bc-4c8a-a24c-5f4651c1ce38 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.390988 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.391900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.392183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.392458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] PATH_INFO: `/v3/projects/c2fdbb262d8f40099c1cd17f6c8bf42c/users/081c1c018e624104b8a62bd61682f5f3/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.394634 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2fdbb262d8f40099c1cd17f6c8bf42c, user_id=081c1c018e624104b8a62bd61682f5f3, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.397927 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.423216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1182/2402] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:30 2022] PUT /identity/v3/projects/c2fdbb262d8f40099c1cd17f6c8bf42c/users/081c1c018e624104b8a62bd61682f5f3/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:30.434219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.434442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.434718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.468833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] MFA Rules not processed for user `081c1c018e624104b8a62bd61682f5f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:30.470446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:30.555136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1221/2403] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:30 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:30.568840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.571804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:30.665771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd4731-eb0b-4caf-a9f0-152d16edc531 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.670373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83d6382-27bf-4e9b-92ac-c38ad5b093d5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.671180 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6382-27bf-4e9b-92ac-c38ad5b093d5 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.672126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6382-27bf-4e9b-92ac-c38ad5b093d5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.672126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83d6382-27bf-4e9b-92ac-c38ad5b093d5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.685110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d6382-27bf-4e9b-92ac-c38ad5b093d5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.686456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83d6382-27bf-4e9b-92ac-c38ad5b093d5 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.699675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1183/2404] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:30.874724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:30.876219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:30.888375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dab84af-d523-4fa2-90fc-e4711637baad None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:30.891984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:30.892676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:30.892965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:30.893244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:30.954851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:30.956216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:30.965525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1222/2405] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:30 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:33.203401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:33.204445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:33.204807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:33.226351 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:33.227650 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:33.283018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:33.291052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1184/2406] 10.222.0.70 () {58 vars in 1009 bytes} [Mon May 23 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:33.308238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:33.309732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:33.384899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:33.402863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23fa342a-02db-4f57-a204-3e099a531e4d service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:33.412007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbc1a6a-3c7c-4cde-83ed-02d48960ad69 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:33.418994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1a6a-3c7c-4cde-83ed-02d48960ad69 service swift] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:33.418994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1a6a-3c7c-4cde-83ed-02d48960ad69 service swift] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:33.418994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbc1a6a-3c7c-4cde-83ed-02d48960ad69 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:33.423431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1a6a-3c7c-4cde-83ed-02d48960ad69 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:33.425090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbc1a6a-3c7c-4cde-83ed-02d48960ad69 service swift] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:33.436609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1223/2407] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 14:09:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 23 14:09:36.708252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:36.837819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bfe5b6-2138-433a-8dd4-bdca45e1fccb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:36.842974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:36.843946 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:36.844363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:36.844826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] PATH_INFO: `/v3/users/60f19f40bf324b52b516913de26217b9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:36.846272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] RBAC: Authorizing `identity:delete_user(user_id=60f19f40bf324b52b516913de26217b9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:36.849397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:37.107025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '60f19f40bf324b52b516913de26217b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:37.117032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '60f19f40bf324b52b516913de26217b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:37.134892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '60f19f40bf324b52b516913de26217b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:37.135830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '60f19f40bf324b52b516913de26217b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:37.233537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '60f19f40bf324b52b516913de26217b9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:37.244863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1185/2408] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:36 2022] DELETE /identity/v3/users/60f19f40bf324b52b516913de26217b9 => generated 0 bytes in 538 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:37.250468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:37.251425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:37.251768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:37.271669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:37.272940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:37.339090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1224/2409] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:37.351191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:37.352760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:37.438188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9be8774-7609-4c73-aad8-c1301ed2045d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:37.443157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:37.444078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:37.444366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:37.444676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:37.455888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:37.457475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:37.468471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1186/2410] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:38.774313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:38.776408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:38.803395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2400a7-c718-4d3f-9c70-04e39a880968 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:38.815516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:38.818196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:38.818953 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:38.819494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] PATH_INFO: `/v3/projects/3b9c25999dd3466784c2bd657cd5a147` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:38.833214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9c25999dd3466784c2bd657cd5a147)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:38.839201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:39.028054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9c25999dd3466784c2bd657cd5a147', 'request_id': 'req-92e402d1-32ef-4621-bc91-211ae174bea9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.037033 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9c25999dd3466784c2bd657cd5a147', 'request_id': 'req-92e402d1-32ef-4621-bc91-211ae174bea9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.046534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9c25999dd3466784c2bd657cd5a147', 'request_id': 'req-92e402d1-32ef-4621-bc91-211ae174bea9'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.138029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] The token cache is being invalidate because project 3b9c25999dd3466784c2bd657cd5a147 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:39.138432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.139354 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.141204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1225/2411] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:38 2022] DELETE /identity/v3/projects/3b9c25999dd3466784c2bd657cd5a147 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:39.146138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:39.161089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a0137a9-1773-413b-9cf2-9ad3f3265c0c service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:39.195046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:39.249688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:39.251513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:39.252015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:39.252301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] PATH_INFO: `/v3/users/081c1c018e624104b8a62bd61682f5f3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:39.253683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=081c1c018e624104b8a62bd61682f5f3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:39.255839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:39.472505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '081c1c018e624104b8a62bd61682f5f3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.479651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '081c1c018e624104b8a62bd61682f5f3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.501424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '081c1c018e624104b8a62bd61682f5f3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.502182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '081c1c018e624104b8a62bd61682f5f3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.593408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6c39cefd-aaaa-4518-98d1-c65d4130d729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '081c1c018e624104b8a62bd61682f5f3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:39.606164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1187/2412] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:39 2022] DELETE /identity/v3/users/081c1c018e624104b8a62bd61682f5f3 => generated 0 bytes in 461 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:40.150617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:40.153026 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:40.169223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e402d1-32ef-4621-bc91-211ae174bea9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:40.278065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:40.279056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:40.279341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:40.279620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] PATH_INFO: `/v3/projects/c2fdbb262d8f40099c1cd17f6c8bf42c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:40.280891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2fdbb262d8f40099c1cd17f6c8bf42c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:40.282921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:40.284991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:40.436835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2fdbb262d8f40099c1cd17f6c8bf42c', 'request_id': 'req-444e740d-356d-40f7-8496-7328d480446d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:40.444207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2fdbb262d8f40099c1cd17f6c8bf42c', 'request_id': 'req-444e740d-356d-40f7-8496-7328d480446d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:40.453513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2fdbb262d8f40099c1cd17f6c8bf42c', 'request_id': 'req-444e740d-356d-40f7-8496-7328d480446d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:40.534005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] The token cache is being invalidate because project c2fdbb262d8f40099c1cd17f6c8bf42c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:40.534346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:40.535114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-444e740d-356d-40f7-8496-7328d480446d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:40.537605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1226/2413] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:40 2022] DELETE /identity/v3/projects/c2fdbb262d8f40099c1cd17f6c8bf42c => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:40.542376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:40.543003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:40.543108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:40.574523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:40.576217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:40.700859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1188/2414] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:40.716649 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:40.716933 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:40.717156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:40.738866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:40.742448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:40.752337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1227/2415] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:40.757344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:40.768143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4ca87b-27bf-4293-b74e-4234d8120f8b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:40.774130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:40.775018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:40.775354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:40.775637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:40.776408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:09:40.776789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:40.778472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:40.800745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:40.811181 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57cddb3-537b-4044-a6cf-ded612ab5069 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:40.812078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1189/2416] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:09:40.824283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:40.825885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:40.825885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:40.825885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:40.829609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:40.832107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:40.948755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1228/2417] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:40 2022] POST /identity/v3/projects => generated 385 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:09:40.953485 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:40.965460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2555ed-f154-4282-bdbd-372a8824f6fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:40.996887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:41.030441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.031223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.031529 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.031836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.032754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.034751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.124815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1190/2418] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:40 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:41.129247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.141418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376fee15-5998-4eaf-acb4-d2578c28b0ef admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.146171 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.146836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.147118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.147393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.148146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.149878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.163932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1229/2419] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:09:41.174305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.186241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f077f931-c1d7-43ef-8e9e-2a4353340162 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.191099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8b542d-0bd0-4770-baed-76355f72208e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.192685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b542d-0bd0-4770-baed-76355f72208e admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.193407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b542d-0bd0-4770-baed-76355f72208e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.193407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8b542d-0bd0-4770-baed-76355f72208e admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.193904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b542d-0bd0-4770-baed-76355f72208e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.195595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8b542d-0bd0-4770-baed-76355f72208e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.206971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1191/2420] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:41.211817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.224113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e34538-e455-4efa-807c-950a267501c7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.229552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.230463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.230783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.231066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] PATH_INFO: `/v3/projects/707aaad092ab498280ced6383fd7d331/users/9244860671ba43919c282d0de394802e/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.248707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] RBAC: Authorizing `identity:create_grant(project_id=707aaad092ab498280ced6383fd7d331, user_id=9244860671ba43919c282d0de394802e, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.258385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.365786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1230/2421] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:41 2022] PUT /identity/v3/projects/707aaad092ab498280ced6383fd7d331/users/9244860671ba43919c282d0de394802e/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:41.376404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.376754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.377188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.415143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] MFA Rules not processed for user `9244860671ba43919c282d0de394802e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:41.416457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:41.484807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1192/2422] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:09:41.490085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.556959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3ba39b-b8aa-4016-9d06-19674419a011 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.561967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.562638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.562936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.563212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.563910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.565854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.640601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1231/2423] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:41 2022] POST /identity/v3/projects => generated 387 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:41.644850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.655738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eecc42-ceea-46f7-b71b-093f6feab986 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.722279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.723527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.724470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.724797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.725516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.727507 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.803272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1193/2424] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:41 2022] POST /identity/v3/users => generated 300 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:41.807470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.818317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ce3a7-4637-4c74-b168-28b517249d6a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.823016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.823727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.823978 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.824256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.825030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.826913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.833734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:41.846739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1232/2425] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:41.847177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.860194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44cb5398-8cb6-44f9-9c62-a2a51b276d12 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:41.865858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:41.866616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:41.866908 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:41.867189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] PATH_INFO: `/v3/projects/2aaa81e12a104b3e9d6635bdf533d6af/users/3204f80c910148e596bd3f337e6ee621/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:41.883436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaa81e12a104b3e9d6635bdf533d6af, user_id=3204f80c910148e596bd3f337e6ee621, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:41.886219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:41.958305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1194/2426] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:41 2022] PUT /identity/v3/projects/2aaa81e12a104b3e9d6635bdf533d6af/users/3204f80c910148e596bd3f337e6ee621/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:41.962339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:41.972348 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5814988-40b2-4ca4-8d89-9c31c55bbcf0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:42.041083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f35046-a017-4d88-ac60-c863975dbd7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:42.041963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f35046-a017-4d88-ac60-c863975dbd7d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.042147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f35046-a017-4d88-ac60-c863975dbd7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.042431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f35046-a017-4d88-ac60-c863975dbd7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.043176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f35046-a017-4d88-ac60-c863975dbd7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:42.044825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f35046-a017-4d88-ac60-c863975dbd7d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:42.056276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1233/2427] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:42.060331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:42.066151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:42.071524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa675115-dc8a-4b13-a589-d7692fcb9016 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:42.076464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:42.077689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.077977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.078259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] PATH_INFO: `/v3/projects/2aaa81e12a104b3e9d6635bdf533d6af/users/3204f80c910148e596bd3f337e6ee621/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.080265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2aaa81e12a104b3e9d6635bdf533d6af, user_id=3204f80c910148e596bd3f337e6ee621, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:42.084121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:42.094722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1195/2428] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:42 2022] PUT /identity/v3/projects/2aaa81e12a104b3e9d6635bdf533d6af/users/3204f80c910148e596bd3f337e6ee621/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:42.107471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.107760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.108038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.138007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] MFA Rules not processed for user `3204f80c910148e596bd3f337e6ee621`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:42.139251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:42.215364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1234/2429] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:42 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:42.227527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:42.229441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:42.294498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fbaab9a-4131-4bd3-a53d-31728fac1a91 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:42.298213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ab2a3-1979-45b1-ab41-bead28597629 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:42.298862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab2a3-1979-45b1-ab41-bead28597629 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.299143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab2a3-1979-45b1-ab41-bead28597629 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.299416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ab2a3-1979-45b1-ab41-bead28597629 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.309040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ab2a3-1979-45b1-ab41-bead28597629 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:42.310404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ab2a3-1979-45b1-ab41-bead28597629 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:42.321029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1196/2430] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:42 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:42.392918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:42.461645 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f655f4-c621-4e3f-80fb-45567c4f8bab None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:42.466310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:42.467270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.467551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.467824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] PATH_INFO: `/v3/users/9244860671ba43919c282d0de394802e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.469115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9244860671ba43919c282d0de394802e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:42.471038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:42.495686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:42.728805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9244860671ba43919c282d0de394802e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:42.729574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9244860671ba43919c282d0de394802e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:42.731292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9244860671ba43919c282d0de394802e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:42.746241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9244860671ba43919c282d0de394802e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:42.753572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9244860671ba43919c282d0de394802e'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:42.810957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1235/2431] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:42 2022] DELETE /identity/v3/users/9244860671ba43919c282d0de394802e => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:42.815562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.816064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.816195 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.835618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:42.836874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:42.907020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1197/2432] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:42.917979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:42.919421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:42.980882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bc4c305-abab-4f09-88df-8aa91a2cfcb8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:42.984469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:42.985167 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:42.985458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:42.985744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:42.996731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:42.998205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:43.008674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1236/2433] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:43.503155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:43.504625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:43.516674 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dd73ea-6d6d-4546-8b45-2b780e4a126e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:43.522014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:43.522884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:43.523197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:43.523472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] PATH_INFO: `/v3/projects/707aaad092ab498280ced6383fd7d331` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:43.524701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=707aaad092ab498280ced6383fd7d331)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:43.526723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:43.673447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '707aaad092ab498280ced6383fd7d331', 'request_id': 'req-5694b6d6-4437-430b-99bf-9c3a050e77c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:43.681917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '707aaad092ab498280ced6383fd7d331', 'request_id': 'req-5694b6d6-4437-430b-99bf-9c3a050e77c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:43.688512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '707aaad092ab498280ced6383fd7d331', 'request_id': 'req-5694b6d6-4437-430b-99bf-9c3a050e77c1'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:43.776515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] The token cache is being invalidate because project 707aaad092ab498280ced6383fd7d331 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:43.776768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:43.777334 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:43.779000 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1198/2434] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:43 2022] DELETE /identity/v3/projects/707aaad092ab498280ced6383fd7d331 => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:43.783183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:43.793342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1b5b58-0333-492d-a144-02513cb2b72f service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:43.851232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:43.852122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:43.852429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:43.852738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] PATH_INFO: `/v3/users/3204f80c910148e596bd3f337e6ee621` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:43.853934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3204f80c910148e596bd3f337e6ee621)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:43.855864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:44.041915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:44.043592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3204f80c910148e596bd3f337e6ee621'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.044369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3204f80c910148e596bd3f337e6ee621'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.045539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3204f80c910148e596bd3f337e6ee621'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.060193 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3204f80c910148e596bd3f337e6ee621'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.068030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-124da513-c977-465a-be0e-ba1ff9552793 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3204f80c910148e596bd3f337e6ee621'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.137916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1237/2435] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:43 2022] DELETE /identity/v3/users/3204f80c910148e596bd3f337e6ee621 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:44.624025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:44.626159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:44.646343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5694b6d6-4437-430b-99bf-9c3a050e77c1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:44.662430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:44.711511 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:44.712490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:44.712857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:44.713168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] PATH_INFO: `/v3/projects/2aaa81e12a104b3e9d6635bdf533d6af` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:44.714374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=2aaa81e12a104b3e9d6635bdf533d6af)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:44.716327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:44.856374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2aaa81e12a104b3e9d6635bdf533d6af', 'request_id': 'req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.862874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2aaa81e12a104b3e9d6635bdf533d6af', 'request_id': 'req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.869222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2aaa81e12a104b3e9d6635bdf533d6af', 'request_id': 'req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.970420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] The token cache is being invalidate because project 2aaa81e12a104b3e9d6635bdf533d6af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:44.971109 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.971947 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-2c8beeff-9d9f-409e-af0d-8fe9a691d0ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:44.974526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1199/2436] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:44 2022] DELETE /identity/v3/projects/2aaa81e12a104b3e9d6635bdf533d6af => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:44.979714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:44.980263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:44.980263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:44.999437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:45.000694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:45.062590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1238/2437] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:45.071692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.071980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.072258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.093058 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:45.094329 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:45.103085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1200/2438] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:45.107562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.118111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3b8c3e-fd39-4fca-95a3-33ac5e3305dd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.122842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.123521 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.123746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.124016 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.124883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:09:45.125189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.126792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.139738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1239/2439] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:09:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:09:45.144369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.154777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c764c0ab-1b87-4a26-a6aa-2c9d3f36ea56 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.159746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.160487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.161420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.161710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.162427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.164466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.267094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1201/2440] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:45 2022] POST /identity/v3/projects => generated 399 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:45.272339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.283526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614b61de-4c27-4cb4-ada1-573fe34d1ba4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.345558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.346258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.346553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.346820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.347527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.349769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.451495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1240/2441] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:45 2022] POST /identity/v3/users => generated 300 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:45.455751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.467657 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068f9b5d-d6cc-498c-b624-0e273545200a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.472502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.473526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.474288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.474576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.475339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.477032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.491305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1202/2442] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:09:45.495266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.506446 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26565386-c51c-48be-8b11-d45350562acf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.511598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2078a02b-a04e-4e1d-905c-4e192da23798 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.512296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078a02b-a04e-4e1d-905c-4e192da23798 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.512597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078a02b-a04e-4e1d-905c-4e192da23798 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.512901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2078a02b-a04e-4e1d-905c-4e192da23798 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.513641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078a02b-a04e-4e1d-905c-4e192da23798 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.515260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2078a02b-a04e-4e1d-905c-4e192da23798 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.527526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1241/2443] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:45 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:45.531762 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.543695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2d3999-cc53-4e71-a45f-b107d8f35f59 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.549003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.554560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.554560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.554560 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] PATH_INFO: `/v3/projects/5cc7efd100ef4fd18008c9f0af94e24b/users/2a788e2e2f5d4fdab7a8045a0b692a39/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.578245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5cc7efd100ef4fd18008c9f0af94e24b, user_id=2a788e2e2f5d4fdab7a8045a0b692a39, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.581119 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.653451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1203/2444] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:45 2022] PUT /identity/v3/projects/5cc7efd100ef4fd18008c9f0af94e24b/users/2a788e2e2f5d4fdab7a8045a0b692a39/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:09:45.664762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.665055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.665343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.698854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] MFA Rules not processed for user `2a788e2e2f5d4fdab7a8045a0b692a39`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:45.700156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:45.727078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:45.786154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1242/2445] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:45 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:45.790989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.840968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:45.860099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f610c2c-d82b-474e-877d-33a87c2af1f3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:45.865887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:45.866577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:45.866847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:45.867104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:45.867803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:45.869901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:45.978430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1204/2446] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:45 2022] POST /identity/v3/projects => generated 401 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:45.982737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:45.998849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38b9c57-1493-4505-a27a-28c30739d632 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:46.061192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:46.061887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.062174 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.062459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.063201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:46.065185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:46.130283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1243/2447] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:45 2022] POST /identity/v3/users => generated 307 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:46.138176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:46.149052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd4b71-5e07-4003-9e8c-dfb827f3d6d4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:46.154274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:46.155047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.155306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.155679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.156315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:46.157983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:46.171113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1205/2448] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:46.175384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:46.186876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55b6cb3-86e1-4b1c-b524-0f1071738664 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:46.191770 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:46.193352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.193640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.193925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] PATH_INFO: `/v3/projects/50c9826cfd094a0e84234b0839bad1b0/users/13809cbafcf240ac840d4a7ec9ad998c/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.211437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c9826cfd094a0e84234b0839bad1b0, user_id=13809cbafcf240ac840d4a7ec9ad998c, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:46.214295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:46.358527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1244/2449] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:46 2022] PUT /identity/v3/projects/50c9826cfd094a0e84234b0839bad1b0/users/13809cbafcf240ac840d4a7ec9ad998c/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:46.363868 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:46.376005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74601b9a-80b1-48d1-b3f9-b8ebc75dc52a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:46.443871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cf54ef7-06b3-487d-9094-8a3910caf5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:46.444483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf54ef7-06b3-487d-9094-8a3910caf5a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.444802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf54ef7-06b3-487d-9094-8a3910caf5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.445089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cf54ef7-06b3-487d-9094-8a3910caf5a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.445838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf54ef7-06b3-487d-9094-8a3910caf5a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:46.447472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cf54ef7-06b3-487d-9094-8a3910caf5a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:46.461952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1206/2450] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:46.465819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:46.477165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de25ff-a370-414a-9c56-f0e7187e3f54 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:46.481854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:46.482753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.483075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.483350 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] PATH_INFO: `/v3/projects/50c9826cfd094a0e84234b0839bad1b0/users/13809cbafcf240ac840d4a7ec9ad998c/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.485742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50c9826cfd094a0e84234b0839bad1b0, user_id=13809cbafcf240ac840d4a7ec9ad998c, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:46.488404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:46.500479 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1245/2451] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:46 2022] PUT /identity/v3/projects/50c9826cfd094a0e84234b0839bad1b0/users/13809cbafcf240ac840d4a7ec9ad998c/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:46.511790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.512024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.512265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.542302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] MFA Rules not processed for user `13809cbafcf240ac840d4a7ec9ad998c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:46.543594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:46.621800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1207/2452] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:46 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:46.634859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:46.636843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:46.714102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c407463-9697-40f3-b3e0-cb3aee8cfb85 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:46.721411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:46.722092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:46.722375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:46.722648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:46.733398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:46.734875 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:46.746495 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1246/2453] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:46 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:47.085492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:47.095612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1fc5f4-c255-4a69-87ab-b4209c47ff48 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:47.099878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5db1cd-6b79-4394-b403-e6814078ad82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:47.100614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5db1cd-6b79-4394-b403-e6814078ad82 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:47.100900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5db1cd-6b79-4394-b403-e6814078ad82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:47.101182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5db1cd-6b79-4394-b403-e6814078ad82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:47.141397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:47.167434 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5db1cd-6b79-4394-b403-e6814078ad82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:47.169177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5db1cd-6b79-4394-b403-e6814078ad82 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:47.180557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1208/2454] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:47 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:53.403343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:53.417831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:53.474185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d18de16-d085-44a2-8850-edd2da95782c service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:53.478803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:53.479688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:53.479975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:53.480252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] PATH_INFO: `/v3/users/2a788e2e2f5d4fdab7a8045a0b692a39` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:53.481478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2a788e2e2f5d4fdab7a8045a0b692a39)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:53.485494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:53.619401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2a788e2e2f5d4fdab7a8045a0b692a39'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:53.620148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2a788e2e2f5d4fdab7a8045a0b692a39'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:53.620902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2a788e2e2f5d4fdab7a8045a0b692a39'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:53.634431 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2a788e2e2f5d4fdab7a8045a0b692a39'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:53.641505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2a788e2e2f5d4fdab7a8045a0b692a39'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:53.741125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1247/2455] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:53 2022] DELETE /identity/v3/users/2a788e2e2f5d4fdab7a8045a0b692a39 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:53.745654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:53.746121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:53.746249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:53.764761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:53.766038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:53.844969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1209/2456] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:53.852444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:53.854367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:53.921571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19f6075-b191-43c6-a311-bed3ad33d057 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:53.925459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:53.926704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:53.926984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:53.927779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:53.938041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:53.939547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:53.950313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1248/2457] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:09:54.367072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:54.368891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:54.384228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac561d-81f2-48a1-a081-8dcbf7aa94a4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:54.389186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:54.391867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:54.391867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:54.391867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] PATH_INFO: `/v3/projects/5cc7efd100ef4fd18008c9f0af94e24b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:54.391867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5cc7efd100ef4fd18008c9f0af94e24b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:54.393783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:54.549340 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5cc7efd100ef4fd18008c9f0af94e24b', 'request_id': 'req-c4b369ee-6dc8-4ce4-8ee4-71362371a241'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.563266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5cc7efd100ef4fd18008c9f0af94e24b', 'request_id': 'req-c4b369ee-6dc8-4ce4-8ee4-71362371a241'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.575473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5cc7efd100ef4fd18008c9f0af94e24b', 'request_id': 'req-c4b369ee-6dc8-4ce4-8ee4-71362371a241'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.657558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] The token cache is being invalidate because project 5cc7efd100ef4fd18008c9f0af94e24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:54.657852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.658587 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.660236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1210/2458] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:54 2022] DELETE /identity/v3/projects/5cc7efd100ef4fd18008c9f0af94e24b => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:54.666561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:54.679375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4a0447-80e4-4f38-9fb0-a55e9b916343 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:54.726424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:54.758766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:54.760042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:54.760327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:54.760628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] PATH_INFO: `/v3/users/13809cbafcf240ac840d4a7ec9ad998c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:54.762457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13809cbafcf240ac840d4a7ec9ad998c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:54.765364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:54.965994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13809cbafcf240ac840d4a7ec9ad998c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.965994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13809cbafcf240ac840d4a7ec9ad998c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.965994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13809cbafcf240ac840d4a7ec9ad998c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.985482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13809cbafcf240ac840d4a7ec9ad998c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:54.995054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fb4ac216-be57-425c-a6dd-e753c0e16652 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13809cbafcf240ac840d4a7ec9ad998c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:55.120486 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1249/2459] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:09:54 2022] DELETE /identity/v3/users/13809cbafcf240ac840d4a7ec9ad998c => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:55.595287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:55.599593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:55.611354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b369ee-6dc8-4ce4-8ee4-71362371a241 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:55.684520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:55.685408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:55.685685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:55.685959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] PATH_INFO: `/v3/projects/50c9826cfd094a0e84234b0839bad1b0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:55.687156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50c9826cfd094a0e84234b0839bad1b0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:55.689121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:55.690795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:55.793795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50c9826cfd094a0e84234b0839bad1b0', 'request_id': 'req-e69e84f8-1de7-4131-84a5-4f646df6db74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:55.799929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50c9826cfd094a0e84234b0839bad1b0', 'request_id': 'req-e69e84f8-1de7-4131-84a5-4f646df6db74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:55.806235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50c9826cfd094a0e84234b0839bad1b0', 'request_id': 'req-e69e84f8-1de7-4131-84a5-4f646df6db74'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:55.907253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] The token cache is being invalidate because project 50c9826cfd094a0e84234b0839bad1b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:09:55.907489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:55.908046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e69e84f8-1de7-4131-84a5-4f646df6db74 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:09:55.909648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1211/2460] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:09:55 2022] DELETE /identity/v3/projects/50c9826cfd094a0e84234b0839bad1b0 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:55.914772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:55.915293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:55.915415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:55.933767 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:55.935023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:55.999744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1250/2461] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:09:56.060695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.060978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.061256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.081738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:56.083079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:56.091004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1212/2462] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:56.095330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.105795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1579b0a-eeab-46b7-ae3b-6256cb6460e4 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.110374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.111034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.111323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.111600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.112331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:09:56.112676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.115347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.126897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1251/2463] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:09:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:09:56.131731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.140915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9189564-a028-4846-aa31-cd876efaab40 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.146164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.146870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.147207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.147401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.148083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.150017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.236399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1213/2464] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:56 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:56.241059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.253105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b571eebb-60e8-4982-8783-d9bfaad94b0f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.307266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:56.324053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.324781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.325069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.326959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.326959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.328505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.457098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1252/2465] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:56 2022] POST /identity/v3/users => generated 287 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:09:56.461203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.471583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1116c41-abf3-4e97-a332-9b31cfc7414b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.477005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.477706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.477992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.478271 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.478997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.480635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.491992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1214/2466] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:56.496160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.506546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77e53ef8-b702-453a-abb2-447ee939e84c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.511265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94d03ef3-e6a2-422a-bcbc-a515b11de8f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.511943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d03ef3-e6a2-422a-bcbc-a515b11de8f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.512223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d03ef3-e6a2-422a-bcbc-a515b11de8f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.512506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94d03ef3-e6a2-422a-bcbc-a515b11de8f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.513270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d03ef3-e6a2-422a-bcbc-a515b11de8f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.514877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94d03ef3-e6a2-422a-bcbc-a515b11de8f6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.525185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1253/2467] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:09:56.529458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.540906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2beca165-8b01-4102-95ab-3b49f2ab3bf6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.546050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.546937 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.547220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.547501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] PATH_INFO: `/v3/projects/0cfe2b8dec274a588eab21425e4ed1d6/users/0f2622063f6e4f698549b6e7034e4712/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.563025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0cfe2b8dec274a588eab21425e4ed1d6, user_id=0f2622063f6e4f698549b6e7034e4712, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.565742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.655063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1215/2468] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:56 2022] PUT /identity/v3/projects/0cfe2b8dec274a588eab21425e4ed1d6/users/0f2622063f6e4f698549b6e7034e4712/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:56.665591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.665883 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.666163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.695430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] MFA Rules not processed for user `0f2622063f6e4f698549b6e7034e4712`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:56.696706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:56.763308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1254/2469] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:56 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:56.783164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.855969 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f486ca91-2123-4168-8d6a-df09453bc211 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:56.861110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:56.861894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:56.862184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:56.862487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:56.863184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:56.865327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:56.949300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1216/2470] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:09:56 2022] POST /identity/v3/projects => generated 375 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:56.954636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:56.969103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d76efa-b88b-4816-a23c-bd1b864c7b2d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:57.044369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:57.045239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.045530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.045814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.046530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:57.048453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:57.178023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1255/2471] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:09:56 2022] POST /identity/v3/users => generated 294 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:09:57.182076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:57.193786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d051ab13-6cad-424b-acd5-a878a9de5ac7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:57.198592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:57.199262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.199551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.199831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.200592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:57.202500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:57.209747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:57.215470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1217/2472] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:09:57.234810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:57.245413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd00ef48-2ed0-4c33-80f2-b8b1c646712e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:57.250365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:57.251282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.251560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.251867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] PATH_INFO: `/v3/projects/a4383553f6f54c1ea05fa819a172614a/users/e3d013fdf33843d68683539bf0adefac/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.269779 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4383553f6f54c1ea05fa819a172614a, user_id=e3d013fdf33843d68683539bf0adefac, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:57.273596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:57.451440 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1256/2473] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:57 2022] PUT /identity/v3/projects/a4383553f6f54c1ea05fa819a172614a/users/e3d013fdf33843d68683539bf0adefac/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:57.460304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:57.467114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e604b31b-402f-4232-81ce-3cd2bcca22d9 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:57.532448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b132a58-8a26-4ba5-b38c-a875afcad665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:57.533110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b132a58-8a26-4ba5-b38c-a875afcad665 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.533475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b132a58-8a26-4ba5-b38c-a875afcad665 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.533670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b132a58-8a26-4ba5-b38c-a875afcad665 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.534516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b132a58-8a26-4ba5-b38c-a875afcad665 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:57.536126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b132a58-8a26-4ba5-b38c-a875afcad665 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:57.546810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1218/2474] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:09:57.561507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:57.573266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6211dc94-6b61-44dd-91ce-8ba3ecdbac19 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:57.578109 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:57.578993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.579318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.579546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] PATH_INFO: `/v3/projects/a4383553f6f54c1ea05fa819a172614a/users/e3d013fdf33843d68683539bf0adefac/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.581547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4383553f6f54c1ea05fa819a172614a, user_id=e3d013fdf33843d68683539bf0adefac, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:57.584375 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:57.595159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1257/2475] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:09:57 2022] PUT /identity/v3/projects/a4383553f6f54c1ea05fa819a172614a/users/e3d013fdf33843d68683539bf0adefac/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:09:57.615547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.615845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.616133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.645513 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] MFA Rules not processed for user `e3d013fdf33843d68683539bf0adefac`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:09:57.646873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:57.721865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1219/2476] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:09:57 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:09:57.733888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:09:57.735885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:09:57.742580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:09:57.796456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a79784cc-8776-4f1c-ae51-c0d820f2d1d2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:09:57.800132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72b8eaa-ee50-460b-9adb-1f329622f3bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:09:57.800818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b8eaa-ee50-460b-9adb-1f329622f3bb service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:09:57.801099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b8eaa-ee50-460b-9adb-1f329622f3bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:09:57.801937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72b8eaa-ee50-460b-9adb-1f329622f3bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:09:57.870491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b8eaa-ee50-460b-9adb-1f329622f3bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:09:57.873394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72b8eaa-ee50-460b-9adb-1f329622f3bb service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:09:57.886221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1258/2477] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:10:02.264148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:02.358980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7179313b-f449-40e1-9635-eadaae99c9b1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:02.363645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:02.365754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:02.366044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:02.366317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] PATH_INFO: `/v3/users/0f2622063f6e4f698549b6e7034e4712` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:02.367543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f2622063f6e4f698549b6e7034e4712)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:02.369632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:02.392809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:02.648856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f2622063f6e4f698549b6e7034e4712'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:02.658917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f2622063f6e4f698549b6e7034e4712'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:02.672037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f2622063f6e4f698549b6e7034e4712'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:02.672804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f2622063f6e4f698549b6e7034e4712'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:02.771647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f2622063f6e4f698549b6e7034e4712'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:02.778248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1220/2478] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:10:02 2022] DELETE /identity/v3/users/0f2622063f6e4f698549b6e7034e4712 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:10:02.782879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:02.783359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:02.783480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:02.801968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:10:02.803165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:02.863366 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1259/2479] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:10:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:10:02.873294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:02.874710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:02.934411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30215f92-fade-4a36-83be-38e7111833c7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:02.938039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfefd9ac-5d0b-4c52-b6eb-b4188c386484 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:02.938879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd9ac-5d0b-4c52-b6eb-b4188c386484 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:02.939015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd9ac-5d0b-4c52-b6eb-b4188c386484 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:02.939239 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfefd9ac-5d0b-4c52-b6eb-b4188c386484 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:02.950646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd9ac-5d0b-4c52-b6eb-b4188c386484 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:02.952101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfefd9ac-5d0b-4c52-b6eb-b4188c386484 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:02.962660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1221/2480] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:10:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:10:03.492818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:03.494468 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:03.506987 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3c391d-4e2c-4f93-a535-a88b791af798 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:03.511742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:03.512655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:03.512940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:03.513217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] PATH_INFO: `/v3/projects/0cfe2b8dec274a588eab21425e4ed1d6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:03.514414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=0cfe2b8dec274a588eab21425e4ed1d6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:03.516323 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:03.664442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:03.665413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:03.665413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:03.677833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0cfe2b8dec274a588eab21425e4ed1d6', 'request_id': 'req-2420fcb2-808b-4f17-83b0-e17e319c74fc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:03.682333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:03.688414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] MFA Rules not processed for user `c7d9b65b064a4326ad0d8e6235b46fd2`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:10:03.689706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:03.693787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0cfe2b8dec274a588eab21425e4ed1d6', 'request_id': 'req-2420fcb2-808b-4f17-83b0-e17e319c74fc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:03.704279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0cfe2b8dec274a588eab21425e4ed1d6', 'request_id': 'req-2420fcb2-808b-4f17-83b0-e17e319c74fc'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:03.761860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1222/2481] 10.222.0.70 () {58 vars in 967 bytes} [Mon May 23 14:10:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:10:03.778295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:03.779721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:03.780139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] The token cache is being invalidate because project 0cfe2b8dec274a588eab21425e4ed1d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:10:03.780416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:03.781529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:03.783553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1260/2482] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:10:03 2022] DELETE /identity/v3/projects/0cfe2b8dec274a588eab21425e4ed1d6 => generated 0 bytes in 291 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:10:03.787815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:03.789989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:03.844734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:03.872827 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7e5fd5-c363-46b6-9811-a83a8096c37b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:03.878945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e525e14157f94459a6e2ab46ffdeb529', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:03.880845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:03.880845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:03.880845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:03.882751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:03.886949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2420fcb2-808b-4f17-83b0-e17e319c74fc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:03.893028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:03.894617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:03.894793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:03.895116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] PATH_INFO: `/v3/users/e3d013fdf33843d68683539bf0adefac` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:03.898148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e3d013fdf33843d68683539bf0adefac)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:03.899931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:03.900256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:03.902046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:03.935962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1223/2483] 10.222.0.70 () {60 vars in 1384 bytes} [Mon May 23 14:10:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 23 14:10:04.182039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e3d013fdf33843d68683539bf0adefac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:04.182816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e3d013fdf33843d68683539bf0adefac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:04.184103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e3d013fdf33843d68683539bf0adefac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:04.201710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e3d013fdf33843d68683539bf0adefac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:04.212671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-b8e581e3-1e2f-41db-bcd4-4d2ea9cfda25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e3d013fdf33843d68683539bf0adefac'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:04.291054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1261/2484] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:10:03 2022] DELETE /identity/v3/users/e3d013fdf33843d68683539bf0adefac => generated 0 bytes in 504 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:10:04.744536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:04.755837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66745f42-6839-4bf6-92d3-18c5363e0575 service swift] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:04.816794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:04.817817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:04.818025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:04.818291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] PATH_INFO: `/v3/projects/a4383553f6f54c1ea05fa819a172614a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:04.819922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4383553f6f54c1ea05fa819a172614a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:04.822025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.005763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4383553f6f54c1ea05fa819a172614a', 'request_id': 'req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:05.012649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4383553f6f54c1ea05fa819a172614a', 'request_id': 'req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:05.018765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4383553f6f54c1ea05fa819a172614a', 'request_id': 'req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:05.125698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] The token cache is being invalidate because project a4383553f6f54c1ea05fa819a172614a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:10:05.125974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:05.126639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f45be1b5-2ccb-49fd-bdf8-7ccbdbb68744 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:05.128276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1224/2485] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:10:04 2022] DELETE /identity/v3/projects/a4383553f6f54c1ea05fa819a172614a => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:10:05.133692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.134258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.134258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.156239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:10:05.157577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:05.228869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:05.240402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1262/2486] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:10:05.248307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.248628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.248923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.268790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:10:05.271796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:05.285124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1225/2487] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:10:05.293980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:05.307670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a3fce64-4703-47ff-8b9a-a4c20bf9f083 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:05.312785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:05.313491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.314079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.314518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.315288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:10:05.315623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:05.317816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.332407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1263/2488] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:10:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:10:05.337676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:05.352323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec3d121-e2cc-4201-aad2-4366073ad4e6 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:05.357685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:05.358387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.358671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.358946 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.359664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:05.361673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.502174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1226/2489] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:10:05 2022] POST /identity/v3/projects => generated 383 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:10:05.508476 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:05.519072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc40033-6fe8-4a3c-80f7-9d9078132bcf admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:05.585282 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:05.586111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.586411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.586686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.587429 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:05.589422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.685231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1264/2490] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:10:05 2022] POST /identity/v3/users => generated 292 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:10:05.689397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:05.700835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3783b718-3d0f-4062-bd30-f68b9a554ecc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:05.705758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:05.706469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.706865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.707028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.708325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:05.710205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.723447 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1227/2491] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:10:05.728516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:05.739903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf267c2-0da7-47b1-af45-4f20d120665f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:05.745093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e23a0e3b-d6f7-4500-828b-1f55d6a26c87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:05.745786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a0e3b-d6f7-4500-828b-1f55d6a26c87 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.746228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a0e3b-d6f7-4500-828b-1f55d6a26c87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.746397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e23a0e3b-d6f7-4500-828b-1f55d6a26c87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.747137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a0e3b-d6f7-4500-828b-1f55d6a26c87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:05.748783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e23a0e3b-d6f7-4500-828b-1f55d6a26c87 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.764189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1265/2492] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:10:05.766255 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:05.777433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a02646a-c9be-4a66-8914-6cd9bcd1f3dd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:05.782263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:05.783164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.783451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.783731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] PATH_INFO: `/v3/projects/7c607c0307e84336a447dafbcd509996/users/cec85c828fd443678f09e23cd6b6dc12/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.799821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c607c0307e84336a447dafbcd509996, user_id=cec85c828fd443678f09e23cd6b6dc12, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:05.803206 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:05.891175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1228/2493] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:10:05 2022] PUT /identity/v3/projects/7c607c0307e84336a447dafbcd509996/users/cec85c828fd443678f09e23cd6b6dc12/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:10:05.902164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:05.902641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:05.902934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:05.932751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] MFA Rules not processed for user `cec85c828fd443678f09e23cd6b6dc12`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:10:05.934034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:06.000878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1266/2494] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:10:05 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:10:06.014292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:06.016104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:06.023050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:06.074884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-409b5d0c-8bcd-4ec4-a796-1f1ad696ba82 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:06.078847 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c171cd8c-9c8b-49ef-b1b8-161c76714d54 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:06.079500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171cd8c-9c8b-49ef-b1b8-161c76714d54 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:06.079783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171cd8c-9c8b-49ef-b1b8-161c76714d54 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:06.080060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c171cd8c-9c8b-49ef-b1b8-161c76714d54 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:06.089806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171cd8c-9c8b-49ef-b1b8-161c76714d54 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:06.091266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c171cd8c-9c8b-49ef-b1b8-161c76714d54 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:06.101719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1229/2495] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:10:06 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:10:10.443936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:10.503266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:10.509588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6c3f6f-ba31-470a-9ca7-019790e8ce7b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:10.514242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:10.521511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:10.521511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:10.521511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] PATH_INFO: `/v3/users/cec85c828fd443678f09e23cd6b6dc12` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:10.521511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=cec85c828fd443678f09e23cd6b6dc12)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:10.521511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:10.736437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cec85c828fd443678f09e23cd6b6dc12'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:10.737184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cec85c828fd443678f09e23cd6b6dc12'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:10.737820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cec85c828fd443678f09e23cd6b6dc12'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:10.751008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cec85c828fd443678f09e23cd6b6dc12'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:10.758888 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cec85c828fd443678f09e23cd6b6dc12'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:10.862400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1267/2496] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:10:10 2022] DELETE /identity/v3/users/cec85c828fd443678f09e23cd6b6dc12 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:10:10.867224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:10.871509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:10.871509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:10.888754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:10:10.890033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:10.954998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1230/2497] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:10:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:10:10.966028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:10.967415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:11.033665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e120620-f6a4-4fa3-ba5e-3f3347560c2c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:11.037982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:11.038673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:11.038957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:11.039230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:11.049845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:11.051318 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:11.062540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1268/2498] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:10:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:10:11.520017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:11.521541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:11.534344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa5f796-31ab-4a2d-9ae9-ac89ae2c7546 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:11.539795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:11.540740 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:11.541029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:11.541310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] PATH_INFO: `/v3/projects/7c607c0307e84336a447dafbcd509996` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:11.542547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c607c0307e84336a447dafbcd509996)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:11.544493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:11.669153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c607c0307e84336a447dafbcd509996', 'request_id': 'req-6a890055-cbd6-4898-992b-098b849387d7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:11.675711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c607c0307e84336a447dafbcd509996', 'request_id': 'req-6a890055-cbd6-4898-992b-098b849387d7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:11.681901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c607c0307e84336a447dafbcd509996', 'request_id': 'req-6a890055-cbd6-4898-992b-098b849387d7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:11.771174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] The token cache is being invalidate because project 7c607c0307e84336a447dafbcd509996 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:10:11.771465 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:11.772151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:10:11.773777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1231/2499] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:10:11 2022] DELETE /identity/v3/projects/7c607c0307e84336a447dafbcd509996 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:10:21.057861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=54,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:21.767597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=78,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:10:51.592650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:51.596523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:51.652984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec34c90-8a10-4557-9765-31cfe2794142 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:51.701480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:51.704077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:51.704077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:51.704077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:51.704859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:51.779141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:51.780603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:51.791841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1269/2500] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:10:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:10:54.981844 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:10:54.983406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:55.061086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a890055-cbd6-4898-992b-098b849387d7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:10:55.066406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:10:55.067123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:10:55.067409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:10:55.067690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:10:55.069506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:10:55.082388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:10:55.086874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:10:55.094129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1232/2501] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:10:54 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:01.787017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=74 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:05.090034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=32 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:12.059968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:12.061472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:12.129309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72669a8-9102-4ebb-afaa-ccb72be23cbf service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:12.133324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:12.133993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:12.134785 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:12.135059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:12.137069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:12.226115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:12.227646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:12.239143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1270/2502] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:16.053802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:16.053802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:16.067110 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ad149a-e70f-4662-81e6-f4c135d888d2 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:16.071967 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:16.072822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:16.073172 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:16.073456 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:16.075919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:16.097687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:16.100509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:16.120870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1233/2503] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:16.283123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:16.284387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:16.298670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a903431-e7f7-4dda-bbae-c9445ae91515 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:16.303613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:16.304520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:16.304836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:16.305232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] PATH_INFO: `/v3/users/b3eae736c69843ceadc6cbdb258b7195` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:16.327254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3eae736c69843ceadc6cbdb258b7195)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:16.329709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:16.525152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3eae736c69843ceadc6cbdb258b7195'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:16.526009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3eae736c69843ceadc6cbdb258b7195'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:16.526648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3eae736c69843ceadc6cbdb258b7195'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:16.541270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3eae736c69843ceadc6cbdb258b7195'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:16.548971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3cfcfd50-52f2-4bd2-9eee-9f3727be2972 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3eae736c69843ceadc6cbdb258b7195'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:16.624970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1271/2504] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:16 2022] DELETE /identity/v3/users/b3eae736c69843ceadc6cbdb258b7195 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:16.911608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:16.924887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0a71092-93ce-47f9-87fb-0072f4aaedb0 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:17.019135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:17.020711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.021231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.021759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] PATH_INFO: `/v3/projects/611e3a80618d4d4dbfb150456f0d3502` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.025335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] RBAC: Authorizing `identity:delete_project(project_id=611e3a80618d4d4dbfb150456f0d3502)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:17.027924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:17.187123 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '611e3a80618d4d4dbfb150456f0d3502', 'request_id': 'req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:17.198834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '611e3a80618d4d4dbfb150456f0d3502', 'request_id': 'req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:17.214987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '611e3a80618d4d4dbfb150456f0d3502', 'request_id': 'req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:17.637501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] The token cache is being invalidate because project 611e3a80618d4d4dbfb150456f0d3502 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:17.637501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:17.637501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b7df6fc2-446f-47f3-b5d5-5f1c1eb7537e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:17.637501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1234/2505] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:16 2022] DELETE /identity/v3/projects/611e3a80618d4d4dbfb150456f0d3502 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:17.638555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.638555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.638555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.638555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:17.638555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:17.638555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:17.639261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1272/2506] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:17.639436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.639436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.639436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.639436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:17.639436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:17.639436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1235/2507] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:17.639762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:17.639762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1532b9e4-2cae-40f7-8e40-beb061744deb None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:17.639762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:17.639762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.640044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.640044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.640044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:17.640044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:17.640044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:17.640044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1273/2508] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:11:17.640409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:17.640409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378742a4-81b2-4830-b4fb-41a569a5a956 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:17.640409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:17.640409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.640716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.640716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.640716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:17.640716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:17.751790 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1236/2509] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:17 2022] POST /identity/v3/projects => generated 373 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:17.756867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:17.767939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37cbbd1-30f0-424a-8dda-25adbe20a96e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:17.832734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:17.833454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.833721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.834002 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.834733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:17.836709 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:17.956122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1274/2510] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:17 2022] POST /identity/v3/users => generated 287 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:17.963003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:17.975072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70acfd88-69a9-411f-995f-8509620c9728 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:17.984406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:17.985759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:17.986464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:17.986996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:17.988284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:17.991150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:18.010350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1237/2511] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:17 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:18.014718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:18.031444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd796d19-878e-426e-bb95-808e1022af19 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:18.037741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a821089-0f68-4741-87b0-bd7a8992ae16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:18.038603 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821089-0f68-4741-87b0-bd7a8992ae16 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:18.038896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821089-0f68-4741-87b0-bd7a8992ae16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:18.039182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a821089-0f68-4741-87b0-bd7a8992ae16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:18.040003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a821089-0f68-4741-87b0-bd7a8992ae16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:18.042487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a821089-0f68-4741-87b0-bd7a8992ae16 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:18.055625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1275/2512] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:18.060789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:18.072545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17947065-044e-4007-b2cc-bcedad943deb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:18.079176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:18.080312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:18.080771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:18.081170 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] PATH_INFO: `/v3/projects/dcc56cd45b8c4628a780d19b2dacf4b7/users/94730ef714c448b49005bbf2e3cba47a/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:18.127257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcc56cd45b8c4628a780d19b2dacf4b7, user_id=94730ef714c448b49005bbf2e3cba47a, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:18.131956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:18.211970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1238/2513] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:18 2022] PUT /identity/v3/projects/dcc56cd45b8c4628a780d19b2dacf4b7/users/94730ef714c448b49005bbf2e3cba47a/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:18.222541 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:18.222833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:18.223322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:18.253355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] MFA Rules not processed for user `94730ef714c448b49005bbf2e3cba47a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:18.254613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:18.347653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1276/2514] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:18 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:18.360835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:18.364193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:18.392533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:18.470450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2de3e06-6dd6-4c4e-97cd-aa51d921011b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:18.477072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442211-df0c-4f72-997a-661c43006784 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:18.477905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442211-df0c-4f72-997a-661c43006784 service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:18.478297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442211-df0c-4f72-997a-661c43006784 service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:18.478683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89442211-df0c-4f72-997a-661c43006784 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:18.493289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442211-df0c-4f72-997a-661c43006784 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:18.495654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89442211-df0c-4f72-997a-661c43006784 service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:18.514062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1239/2515] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:18 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:20.574574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:20.576285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:20.591547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:20.650672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28191a81-53ce-4274-a0ff-466c336e14c8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:20.655128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:20.656037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:20.656327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:20.656664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] PATH_INFO: `/v3/users/cf30fa512142489a8dea40ce426a5f36` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:20.657857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf30fa512142489a8dea40ce426a5f36)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:20.659829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:20.886117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf30fa512142489a8dea40ce426a5f36'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:20.886837 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf30fa512142489a8dea40ce426a5f36'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:20.887627 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf30fa512142489a8dea40ce426a5f36'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:20.902312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf30fa512142489a8dea40ce426a5f36'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:20.910054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5ab301a7-c32f-405b-bca9-09ef2168f319 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf30fa512142489a8dea40ce426a5f36'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:21.131757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1277/2516] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:20 2022] DELETE /identity/v3/users/cf30fa512142489a8dea40ce426a5f36 => generated 0 bytes in 558 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:21.457254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442211-df0c-4f72-997a-661c43006784 service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:21.469428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89442211-df0c-4f72-997a-661c43006784 service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:21.543208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:21.544090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:21.544371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:21.544703 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] PATH_INFO: `/v3/projects/7649fa9a0ae4411aabf060f99e691520` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:21.545933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=7649fa9a0ae4411aabf060f99e691520)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:21.547926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:21.695452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7649fa9a0ae4411aabf060f99e691520', 'request_id': 'req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:21.703070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7649fa9a0ae4411aabf060f99e691520', 'request_id': 'req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:21.709422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7649fa9a0ae4411aabf060f99e691520', 'request_id': 'req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:21.795464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] The token cache is being invalidate because project 7649fa9a0ae4411aabf060f99e691520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:21.795754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:21.796394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:21.798042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1240/2517] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:21 2022] DELETE /identity/v3/projects/7649fa9a0ae4411aabf060f99e691520 => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:21.803239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee9e2-54ff-4dbe-834b-df60b6e22f26 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:21.803703 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee9e2-54ff-4dbe-834b-df60b6e22f26 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:21.803802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeee9e2-54ff-4dbe-834b-df60b6e22f26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:21.822968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ddeee9e2-54ff-4dbe-834b-df60b6e22f26 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:21.824180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ddeee9e2-54ff-4dbe-834b-df60b6e22f26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:21.878454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:21.879949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:21.900923 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1278/2518] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:21.909103 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:21.909407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:21.909748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:21.914253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:21.929952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:21.931982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:21.940471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1279/2519] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:21.945163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:21.950571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a0f4d-a3c5-4c2c-9c90-bbbb018e61ec admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:21.955614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc35e6c3-f09a-4dc9-aa09-73b3d3c0af8c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:21.956613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:21.957566 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:21.957883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:21.958169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] PATH_INFO: `/v3/users/94730ef714c448b49005bbf2e3cba47a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:21.959379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=94730ef714c448b49005bbf2e3cba47a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:21.960337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:21.961043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:21.961328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:21.963913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:21.964023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:21.964023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:21.964023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:21.964273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:21.981052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1280/2520] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 14:11:21.982418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:21.994653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9988b34-d762-45ac-97f5-218e73d129c9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:21.999684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.000300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.000604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.000887 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.001582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.003826 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.007997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:22.152699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1281/2521] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:21 2022] POST /identity/v3/projects => generated 367 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:22.156607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:22.221247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad817bb-0dfe-4d0e-bcf4-4be3cb29696c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:22.232368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.233107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.233396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.233683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.234428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.236707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.240118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '94730ef714c448b49005bbf2e3cba47a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:22.247576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '94730ef714c448b49005bbf2e3cba47a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:22.260325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '94730ef714c448b49005bbf2e3cba47a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:22.261081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '94730ef714c448b49005bbf2e3cba47a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:22.332295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-530eca14-7dd9-45db-aac1-fae3ecc4a1b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '94730ef714c448b49005bbf2e3cba47a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:22.339799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1282/2522] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:22 2022] POST /identity/v3/users => generated 284 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:22.344023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:22.347635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1241/2523] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:21 2022] DELETE /identity/v3/users/94730ef714c448b49005bbf2e3cba47a => generated 0 bytes in 469 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:22.347781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:22.352005 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f48705-f8d0-4842-b557-281155d3621b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.352517 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f48705-f8d0-4842-b557-281155d3621b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.356280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f48705-f8d0-4842-b557-281155d3621b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.361650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11701f42-a2fd-416c-afb5-9c206ecac95c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:22.380422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-67f48705-f8d0-4842-b557-281155d3621b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:22.381741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-67f48705-f8d0-4842-b557-281155d3621b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:22.446087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.447357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.447681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.448013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.448787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.450438 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.459063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1242/2524] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:22.464050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1283/2525] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:22.465926 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f48705-f8d0-4842-b557-281155d3621b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:22.469372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:22.471236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:22.478627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f48705-f8d0-4842-b557-281155d3621b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:22.485244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.485932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.486219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.486498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.487244 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.488919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.501211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1243/2526] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:22 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:22.505667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:22.523310 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6a9fc4-b5a6-4691-a060-e59dd28bc4fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:22.531299 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.533774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.533774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.534065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] PATH_INFO: `/v3/projects/08acab2b5d19419cb7ac3e31e69713bd/users/12842b85187946bc86a49a32a130284b/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.555068 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08acab2b5d19419cb7ac3e31e69713bd, user_id=12842b85187946bc86a49a32a130284b, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.561289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.565218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5088b2fd-084e-4271-8657-d7d3431a6717 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:22.569061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab05cd83-5d3a-4e02-9e05-a2bc4fbcf0b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.569872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05cd83-5d3a-4e02-9e05-a2bc4fbcf0b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.570165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05cd83-5d3a-4e02-9e05-a2bc4fbcf0b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.570439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab05cd83-5d3a-4e02-9e05-a2bc4fbcf0b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.589227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05cd83-5d3a-4e02-9e05-a2bc4fbcf0b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.592131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab05cd83-5d3a-4e02-9e05-a2bc4fbcf0b6 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.609252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1284/2527] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:22.630232 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1244/2528] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:22 2022] PUT /identity/v3/projects/08acab2b5d19419cb7ac3e31e69713bd/users/12842b85187946bc86a49a32a130284b/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:22.642517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.642706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.642970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.672162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:22.677070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] MFA Rules not processed for user `12842b85187946bc86a49a32a130284b`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:22.678319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:22.755965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1285/2529] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:22 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:22.768377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:22.770374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:22.786291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:22.839067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee290644-aca6-4725-a023-34ed69f738f0 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:22.843698 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741032bc-3a42-4a6c-8e5a-e4005322b18f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:22.844776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741032bc-3a42-4a6c-8e5a-e4005322b18f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:22.845107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741032bc-3a42-4a6c-8e5a-e4005322b18f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:22.845405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741032bc-3a42-4a6c-8e5a-e4005322b18f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:22.856284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741032bc-3a42-4a6c-8e5a-e4005322b18f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:22.858641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741032bc-3a42-4a6c-8e5a-e4005322b18f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:22.872555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1245/2530] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:23.235170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:23.237530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:23.326611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c907f8e-ed6d-49e1-90cb-ed92080ddbf7 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:23.331707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:23.332660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:23.332948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:23.333228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] PATH_INFO: `/v3/projects/dcc56cd45b8c4628a780d19b2dacf4b7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:23.334469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcc56cd45b8c4628a780d19b2dacf4b7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:23.336432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:23.477408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcc56cd45b8c4628a780d19b2dacf4b7', 'request_id': 'req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:23.484726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcc56cd45b8c4628a780d19b2dacf4b7', 'request_id': 'req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:23.490872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcc56cd45b8c4628a780d19b2dacf4b7', 'request_id': 'req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:23.573461 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] The token cache is being invalidate because project dcc56cd45b8c4628a780d19b2dacf4b7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:23.573752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:23.574459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e7f25f4c-e7b7-460b-b9e7-ef43fec25bba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:23.576102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1286/2531] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:23 2022] DELETE /identity/v3/projects/dcc56cd45b8c4628a780d19b2dacf4b7 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:23.583756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:23.584272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:23.584681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:23.607711 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:23.611281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:23.674190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1246/2532] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:23.684005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:23.684290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:23.684593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:23.712471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:23.714570 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:23.727546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1287/2533] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:23.733893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:23.749038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8da204-6a86-44fd-a0a9-a6248ba28116 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:23.756095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:23.757011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:23.757429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:23.757818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:23.758752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:23.760129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:23.762710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:23.775839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1247/2534] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 14:11:23.786039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:23.796401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98126c29-557d-4994-9fed-e95204fd6894 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:23.801199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:23.801876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:23.802158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:23.802433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:23.803135 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:23.805355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:23.935587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1288/2535] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:23 2022] POST /identity/v3/projects => generated 363 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:23.940205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:23.951373 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fbe319-8138-4812-9448-11a7d512b1c4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.031132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.031821 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.032107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.032381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.033137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.035217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.130198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1248/2536] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:23 2022] POST /identity/v3/users => generated 282 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:24.135699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.143710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:24.148754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb351fc-3bb5-4e9a-a0ec-05a20415866a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.154336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.155057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.155526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.155652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.156403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.158086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.171121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1289/2537] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:24.175476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.188326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e3ed48-d3e4-412a-9dab-4b388b695526 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.193601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fc734d-1774-4df6-bd48-a849a2d9cda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.194295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc734d-1774-4df6-bd48-a849a2d9cda1 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.194582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc734d-1774-4df6-bd48-a849a2d9cda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.194858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fc734d-1774-4df6-bd48-a849a2d9cda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.195671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc734d-1774-4df6-bd48-a849a2d9cda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.197402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fc734d-1774-4df6-bd48-a849a2d9cda1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.209915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1249/2538] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:24.214615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.225611 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-258295ba-4ca6-45b2-9ddc-e0f954a54aea admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.231377 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.232304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.232614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.232898 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] PATH_INFO: `/v3/projects/34df2e86107444a4a987d5ec3890195a/users/4a64609339b54756a50971f205bd3ef8/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.250690 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34df2e86107444a4a987d5ec3890195a, user_id=4a64609339b54756a50971f205bd3ef8, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.253467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.374621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1290/2539] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:24 2022] PUT /identity/v3/projects/34df2e86107444a4a987d5ec3890195a/users/4a64609339b54756a50971f205bd3ef8/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:24.386660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.387042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.387382 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.419368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] MFA Rules not processed for user `4a64609339b54756a50971f205bd3ef8`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:24.420829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:24.431989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:24.493210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1250/2540] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:24 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:24.499057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.589484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b41eff-2b66-40ec-8636-c7e1b220a301 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.597056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.597753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.598041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.598330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.599045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.601788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.696775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1291/2541] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:24 2022] POST /identity/v3/projects => generated 365 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:24.703376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.715686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a16565-8fcd-4e1c-a241-0e8b7f1c7b85 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.785786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.786489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.786779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.787103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.787833 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.789934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.892599 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1251/2542] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:24 2022] POST /identity/v3/users => generated 289 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:24.898775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.917369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0047c727-b667-4d15-8c57-de5829a6fcf9 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.929831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.930793 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.932852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.933029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:24.933836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:24.935626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:24.957019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1292/2543] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:24 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:24.961404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:24.974205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1aa802-2f01-4c75-a8ce-5bda9baefb37 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:24.982669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:24.983778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:24.984189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:24.984676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] PATH_INFO: `/v3/projects/94b69d9c82d34216b105794f2ccce65b/users/c60e2623aa5c43ad92e6c0d3a2c4eaf7/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:25.006240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b69d9c82d34216b105794f2ccce65b, user_id=c60e2623aa5c43ad92e6c0d3a2c4eaf7, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:25.011092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:25.104490 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1252/2544] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:24 2022] PUT /identity/v3/projects/94b69d9c82d34216b105794f2ccce65b/users/c60e2623aa5c43ad92e6c0d3a2c4eaf7/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:25.108961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:25.121068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30b8e413-7d6f-472d-a112-0760bf3874ae admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:25.192896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32154385-ae69-49a8-9360-f835d47a597b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:25.193619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32154385-ae69-49a8-9360-f835d47a597b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:25.193902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32154385-ae69-49a8-9360-f835d47a597b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:25.194178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32154385-ae69-49a8-9360-f835d47a597b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:25.194909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32154385-ae69-49a8-9360-f835d47a597b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:25.196502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32154385-ae69-49a8-9360-f835d47a597b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:25.207825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1293/2545] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:25 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:25.211986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:25.224660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70f84fc-34d1-49a7-bef7-9b1bb9ca93d1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:25.231817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:25.232777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:25.233066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:25.233339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] PATH_INFO: `/v3/projects/94b69d9c82d34216b105794f2ccce65b/users/c60e2623aa5c43ad92e6c0d3a2c4eaf7/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:25.235631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] RBAC: Authorizing `identity:create_grant(project_id=94b69d9c82d34216b105794f2ccce65b, user_id=c60e2623aa5c43ad92e6c0d3a2c4eaf7, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:25.238359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:25.250038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1253/2546] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:25 2022] PUT /identity/v3/projects/94b69d9c82d34216b105794f2ccce65b/users/c60e2623aa5c43ad92e6c0d3a2c4eaf7/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:25.270040 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:25.270569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:25.270569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:25.309575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] MFA Rules not processed for user `c60e2623aa5c43ad92e6c0d3a2c4eaf7`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:25.310813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:25.347720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:25.392032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1294/2547] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:25 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:25.403130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:25.405020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:25.468245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56e6185-cdb0-42cc-9a10-87c61274285f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:25.471901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:25.472621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:25.472912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:25.473198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:25.486231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:25.488079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:25.500118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1254/2548] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:25 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:25.799229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:25.800693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:25.814727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0275ded-771b-4ff5-99c7-03226ce6e3f6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:25.818597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee737e8a-4715-4b6e-bde4-b31165485550 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:25.819261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e8a-4715-4b6e-bde4-b31165485550 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:25.819546 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e8a-4715-4b6e-bde4-b31165485550 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:25.819824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee737e8a-4715-4b6e-bde4-b31165485550 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:25.892093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee737e8a-4715-4b6e-bde4-b31165485550 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:25.895290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee737e8a-4715-4b6e-bde4-b31165485550 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:25.914021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1295/2549] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:25 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:29.674258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:29.684267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:29.739996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54f7d7-9201-4a7f-9400-f6e207759e6d service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:29.746489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:29.747371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:29.747661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:29.747944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] PATH_INFO: `/v3/users/4a64609339b54756a50971f205bd3ef8` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:29.749177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a64609339b54756a50971f205bd3ef8)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:29.751280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:30.015720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a64609339b54756a50971f205bd3ef8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:30.022488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a64609339b54756a50971f205bd3ef8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:30.034983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a64609339b54756a50971f205bd3ef8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:30.035697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a64609339b54756a50971f205bd3ef8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:30.115509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a64609339b54756a50971f205bd3ef8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:30.124325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1255/2550] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:29 2022] DELETE /identity/v3/users/4a64609339b54756a50971f205bd3ef8 => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:30.129099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:30.129629 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:30.129717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:30.148961 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:30.151410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:30.218498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1296/2551] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:30.228787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:30.230282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:30.293467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427e19fb-ed8b-48eb-bf53-1523be020c0e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:30.297016 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:30.297672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:30.297951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:30.298224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:30.307936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:30.309433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:30.319285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1256/2552] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:30.828137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:30.829909 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:30.854326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56b3db6-b816-41a4-809b-d53ef7820416 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:30.859549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:30.860665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:30.860957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:30.861235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] PATH_INFO: `/v3/projects/34df2e86107444a4a987d5ec3890195a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:30.862498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34df2e86107444a4a987d5ec3890195a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:30.866731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:31.045624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34df2e86107444a4a987d5ec3890195a', 'request_id': 'req-afc17460-8bea-4f23-8c53-50079283b068'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.054467 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34df2e86107444a4a987d5ec3890195a', 'request_id': 'req-afc17460-8bea-4f23-8c53-50079283b068'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.059613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:31.063310 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34df2e86107444a4a987d5ec3890195a', 'request_id': 'req-afc17460-8bea-4f23-8c53-50079283b068'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.195124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] The token cache is being invalidate because project 34df2e86107444a4a987d5ec3890195a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:31.195757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.196048 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.198743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1297/2553] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:30 2022] DELETE /identity/v3/projects/34df2e86107444a4a987d5ec3890195a => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:31.212887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:31.223052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5771cf89-80e4-480e-8def-bef95eb414cf service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:31.252317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:31.285915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:31.286845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:31.287122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:31.287397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] PATH_INFO: `/v3/users/c60e2623aa5c43ad92e6c0d3a2c4eaf7` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:31.288620 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60e2623aa5c43ad92e6c0d3a2c4eaf7)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:31.290530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:31.692767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60e2623aa5c43ad92e6c0d3a2c4eaf7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.701688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60e2623aa5c43ad92e6c0d3a2c4eaf7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.715886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60e2623aa5c43ad92e6c0d3a2c4eaf7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.716706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60e2623aa5c43ad92e6c0d3a2c4eaf7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.812984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-71200da8-81c2-4981-940f-ced3f6005d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60e2623aa5c43ad92e6c0d3a2c4eaf7'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:31.820157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1257/2554] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:31 2022] DELETE /identity/v3/users/c60e2623aa5c43ad92e6c0d3a2c4eaf7 => generated 0 bytes in 608 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:32.375942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:32.378224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:32.393014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc17460-8bea-4f23-8c53-50079283b068 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:32.475248 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:32.476154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:32.476439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:32.476758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] PATH_INFO: `/v3/projects/94b69d9c82d34216b105794f2ccce65b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:32.478081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=94b69d9c82d34216b105794f2ccce65b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:32.480072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:32.604807 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94b69d9c82d34216b105794f2ccce65b', 'request_id': 'req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:32.613123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94b69d9c82d34216b105794f2ccce65b', 'request_id': 'req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:32.620833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94b69d9c82d34216b105794f2ccce65b', 'request_id': 'req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:32.715244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] The token cache is being invalidate because project 94b69d9c82d34216b105794f2ccce65b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:32.715537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:32.716197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4e08dd6b-8530-4a4e-8fbd-edf3050297c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:32.717848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1298/2555] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:32 2022] DELETE /identity/v3/projects/94b69d9c82d34216b105794f2ccce65b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:32.723089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:32.723562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:32.723646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:32.743430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:32.746034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:32.816605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1258/2556] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:32.827487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:32.827683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:32.827969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:32.847355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:32.848604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:32.860423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1299/2557] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:32.864098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:32.885291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bf4ad4-a6fd-461a-bada-a1fc12ef867b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:32.885291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:32.885291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:32.885291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:32.886044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:32.886212 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:32.886676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:32.889279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:32.909595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1259/2558] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:11:32.917212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:32.926079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d28f17d-85d1-44f6-a9dd-c8a5577f8ef8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:32.931698 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:32.932506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:32.932890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:32.933234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:32.934017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:32.936043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:33.061270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1300/2559] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:32 2022] POST /identity/v3/projects => generated 367 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:33.072142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:33.082915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380c0db-aeb7-460e-bf7d-38adbad15418 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:33.130248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:33.165211 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:33.166527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.166829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.167114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.167829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:33.169836 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:33.322175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1260/2560] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:33 2022] POST /identity/v3/users => generated 284 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:33.326814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:33.338233 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb5ef3e-83f5-4b9b-b8f2-cbd5f31262fb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:33.342874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:33.343551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.343835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.344112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.344866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:33.346553 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:33.360769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1301/2561] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:33.364924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:33.376559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532f2b9-95b0-484c-a72a-b537d95536a4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:33.381245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d74dd6-f55d-4aa8-be9b-7d20611b0695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:33.381911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d74dd6-f55d-4aa8-be9b-7d20611b0695 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.382193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d74dd6-f55d-4aa8-be9b-7d20611b0695 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.382468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d74dd6-f55d-4aa8-be9b-7d20611b0695 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.383335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d74dd6-f55d-4aa8-be9b-7d20611b0695 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:33.385205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d74dd6-f55d-4aa8-be9b-7d20611b0695 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:33.395919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1261/2562] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:33.401620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:33.412184 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aba035b-ded1-4e9a-a7af-2fcd65092648 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:33.418528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:33.419374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.419660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.419940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] PATH_INFO: `/v3/projects/d9ba7f4c7bfc4dc59a176d00e0e0e7a4/users/0133a055a18347098a451ddaad1c6300/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.438004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d9ba7f4c7bfc4dc59a176d00e0e0e7a4, user_id=0133a055a18347098a451ddaad1c6300, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:33.440829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:33.557238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1302/2563] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:33 2022] PUT /identity/v3/projects/d9ba7f4c7bfc4dc59a176d00e0e0e7a4/users/0133a055a18347098a451ddaad1c6300/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:33.568115 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.568411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.568736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.601538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] MFA Rules not processed for user `0133a055a18347098a451ddaad1c6300`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:33.602783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:33.673026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1262/2564] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:33 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:33.677975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:33.695211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:33.750798 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b64318-5b0e-4546-b9ea-d8c4115e7214 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:33.755569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:33.756240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.756523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.756827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.757527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:33.760067 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:33.875199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1303/2565] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:33 2022] POST /identity/v3/projects => generated 369 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:33.881838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:33.895416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd82837f-fcf6-4594-802d-e6f6e2f3d621 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:33.959951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:33.960548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:33.960865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:33.961151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:33.961849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:33.963970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:34.095252 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1263/2566] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:33 2022] POST /identity/v3/users => generated 291 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:34.100443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:34.121763 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0655dac4-ef2f-479d-bac6-bdadce6f2c42 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:34.130027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:34.131146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:34.131634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:34.132121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:34.133643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:34.136916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:34.158342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1304/2567] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:34.170800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:34.182999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f5beabb-cd08-46de-b63b-5830e1b49cac admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:34.191651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:34.193305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:34.193791 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:34.194270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] PATH_INFO: `/v3/projects/b1936cebe4264746881ec84f0c67094f/users/b4ef6515afdb4c1a8518d2324497eb0d/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:34.219098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1936cebe4264746881ec84f0c67094f, user_id=b4ef6515afdb4c1a8518d2324497eb0d, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:34.219277 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:34.316361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1264/2568] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:34 2022] PUT /identity/v3/projects/b1936cebe4264746881ec84f0c67094f/users/b4ef6515afdb4c1a8518d2324497eb0d/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:34.320681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:34.332300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5e205f-702c-49b1-a644-df196a5fa844 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:34.398845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1239b3d3-08a7-41ce-93b4-f46d4eb49704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:34.399548 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239b3d3-08a7-41ce-93b4-f46d4eb49704 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:34.399907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239b3d3-08a7-41ce-93b4-f46d4eb49704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:34.400271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1239b3d3-08a7-41ce-93b4-f46d4eb49704 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:34.401051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239b3d3-08a7-41ce-93b4-f46d4eb49704 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:34.402686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1239b3d3-08a7-41ce-93b4-f46d4eb49704 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:34.413675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1305/2569] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:34 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:34.419428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:34.436217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9783c-609e-4699-92bf-4247d90aef70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:34.441396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:34.442315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:34.442596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:34.442870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] PATH_INFO: `/v3/projects/b1936cebe4264746881ec84f0c67094f/users/b4ef6515afdb4c1a8518d2324497eb0d/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:34.445045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=b1936cebe4264746881ec84f0c67094f, user_id=b4ef6515afdb4c1a8518d2324497eb0d, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:34.448732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:34.452337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:34.460183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1265/2570] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:34 2022] PUT /identity/v3/projects/b1936cebe4264746881ec84f0c67094f/users/b4ef6515afdb4c1a8518d2324497eb0d/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:34.471347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:34.471540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:34.471801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:34.502958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] MFA Rules not processed for user `b4ef6515afdb4c1a8518d2324497eb0d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:34.504226 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:34.589676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1306/2571] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:34.602652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:34.605287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:34.671093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068c9539-fafc-422b-8652-bfa263eedf3a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:34.674851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c8592d-09ee-4791-9ef6-e94bce363060 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:34.675505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8592d-09ee-4791-9ef6-e94bce363060 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:34.675794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8592d-09ee-4791-9ef6-e94bce363060 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:34.676133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c8592d-09ee-4791-9ef6-e94bce363060 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:34.687659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8592d-09ee-4791-9ef6-e94bce363060 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:34.689175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c8592d-09ee-4791-9ef6-e94bce363060 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:34.701769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1266/2572] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:35.612621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:35.615057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:35.629983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:35.695636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03fa851-bcd2-45a1-aceb-7d3035b4bc2f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:35.700275 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:35.701189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:35.701473 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:35.701745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] PATH_INFO: `/v3/users/12842b85187946bc86a49a32a130284b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:35.702928 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] RBAC: Authorizing `identity:delete_user(user_id=12842b85187946bc86a49a32a130284b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:35.707726 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:35.941689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12842b85187946bc86a49a32a130284b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:35.942511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12842b85187946bc86a49a32a130284b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:35.943242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12842b85187946bc86a49a32a130284b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:35.958501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12842b85187946bc86a49a32a130284b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:35.966400 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12842b85187946bc86a49a32a130284b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:36.108688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1307/2573] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:35 2022] DELETE /identity/v3/users/12842b85187946bc86a49a32a130284b => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:36.115225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:36.116066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:36.116365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:36.139256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:36.140593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:36.232941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1267/2574] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:36.243316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:36.244894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:36.309681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27959c8c-f0b0-4a37-ac7a-5d09d12b919e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:36.316285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4814c47b-898f-4790-829b-89b416ea1ce5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:36.317867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c47b-898f-4790-829b-89b416ea1ce5 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:36.318130 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c47b-898f-4790-829b-89b416ea1ce5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:36.318614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4814c47b-898f-4790-829b-89b416ea1ce5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:36.331656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c47b-898f-4790-829b-89b416ea1ce5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:36.333579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4814c47b-898f-4790-829b-89b416ea1ce5 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:36.345768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1308/2575] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:36.810018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:36.811871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:36.828358 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffcb5cb-7b68-43c8-ad4b-732ae0db6860 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:36.834048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:36.834694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:36.834971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:36.835385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] PATH_INFO: `/v3/projects/08acab2b5d19419cb7ac3e31e69713bd` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:36.836923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] RBAC: Authorizing `identity:delete_project(project_id=08acab2b5d19419cb7ac3e31e69713bd)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:36.838871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.027727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08acab2b5d19419cb7ac3e31e69713bd', 'request_id': 'req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:37.033956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08acab2b5d19419cb7ac3e31e69713bd', 'request_id': 'req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:37.039669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08acab2b5d19419cb7ac3e31e69713bd', 'request_id': 'req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:37.154766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] The token cache is being invalidate because project 08acab2b5d19419cb7ac3e31e69713bd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:37.154766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:37.154766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fcaeefcb-2c89-4423-8863-f9e87cf8c77e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:37.155897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1268/2576] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:36 2022] DELETE /identity/v3/projects/08acab2b5d19419cb7ac3e31e69713bd => generated 0 bytes in 347 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:37.161183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.161680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.161876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.182121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:37.183691 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:37.217308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:37.259054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1309/2577] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:37.267661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.268128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.268605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.297820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:37.297820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:37.304945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1269/2578] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:37.309405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:37.320111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658508a2-1c47-4a1e-bb50-fd945aeb3dd6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:37.325087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:37.325783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.326068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.326344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.327081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:37.327387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:37.329042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.344316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1310/2579] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:11:37.349342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:37.366993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e751ff8-ae09-4089-a454-dd7e49ae32e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:37.375072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:37.376152 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.376683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.377163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.378399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:37.382149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.401705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:37.479428 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1270/2580] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:37 2022] POST /identity/v3/projects => generated 405 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:37.484239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:37.496735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9c7998-2331-4276-be26-76c42b77f587 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:37.581552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:37.582382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.582802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.583142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.583922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:37.586202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.710897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1311/2581] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:37 2022] POST /identity/v3/users => generated 303 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:37.718370 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:37.733553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d083d52b-c689-4a90-9765-cd524f0a14a1 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:37.739175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:37.739984 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.740359 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.740769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.741635 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:37.744122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.757051 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1271/2582] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:37.762812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:37.773731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54905cfd-3a43-4b8a-889d-0c91791a2f35 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:37.779391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba3146e-6272-4261-ba51-71991d64656a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:37.780131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3146e-6272-4261-ba51-71991d64656a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.780597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3146e-6272-4261-ba51-71991d64656a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.780935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba3146e-6272-4261-ba51-71991d64656a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.781768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3146e-6272-4261-ba51-71991d64656a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:37.783584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba3146e-6272-4261-ba51-71991d64656a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.797361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1312/2583] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:37 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:37.801760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:37.814667 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e0df824-becf-4ffb-b202-1e23d913b7a8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:37.822834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:37.824491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.825020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.825425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] PATH_INFO: `/v3/projects/3f8c11dd14894a3282a593ab7f7c9bf7/users/010c328775884bb19625e1134c2e4277/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.847449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3f8c11dd14894a3282a593ab7f7c9bf7, user_id=010c328775884bb19625e1134c2e4277, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:37.850713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:37.934422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1272/2584] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:37 2022] PUT /identity/v3/projects/3f8c11dd14894a3282a593ab7f7c9bf7/users/010c328775884bb19625e1134c2e4277/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:37.945466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:37.945752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:37.946027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:37.976319 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] MFA Rules not processed for user `010c328775884bb19625e1134c2e4277`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:37.977601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:38.065047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1313/2585] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:37 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:38.069870 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.144601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6307ea6-d161-40df-b048-e70b04b527c8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.150605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.151227 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.151627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.151911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.152678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.154662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.249834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1273/2586] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:38 2022] POST /identity/v3/projects => generated 405 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:38.254515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.266244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee04125e-9d77-4055-91bf-523d9829605a None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.293158 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:38.327967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.328675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.328968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.329240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.330209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.332265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.371676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.374431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:38.394842 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-840323ac-bb52-4b4a-ac70-3dca3148ad67 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.402515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.403498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.403803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.404178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] PATH_INFO: `/v3/users/0133a055a18347098a451ddaad1c6300` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.405653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0133a055a18347098a451ddaad1c6300)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.408736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.455522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1314/2587] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:38 2022] POST /identity/v3/users => generated 309 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:38.460615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.474283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1594f767-14cd-4a26-9909-ff5a0552d00d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.480054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.483827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.483827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.483827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.483827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.485427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.500185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1315/2588] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:38.505878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.517126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73cc3f70-4c93-4e33-8de3-6898775e7e1b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.522110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.523021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.523300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.523573 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] PATH_INFO: `/v3/projects/0b7f6017de924b56962fa9751657dbfc/users/641ad7942b1649528b7103a582016dc8/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.540298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7f6017de924b56962fa9751657dbfc, user_id=641ad7942b1649528b7103a582016dc8, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.544973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.659721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0133a055a18347098a451ddaad1c6300'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:38.668034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0133a055a18347098a451ddaad1c6300'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:38.683278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0133a055a18347098a451ddaad1c6300'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:38.683722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1316/2589] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:38 2022] PUT /identity/v3/projects/0b7f6017de924b56962fa9751657dbfc/users/641ad7942b1649528b7103a582016dc8/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:38.687586 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0133a055a18347098a451ddaad1c6300'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:38.688506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.780244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a9ebda-f3ac-46bb-aa4c-7c927da8e47c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.788871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.790020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.790534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.791415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.792379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.795716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.796140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-cf429373-29f7-4451-b02d-aeef803e1fe9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0133a055a18347098a451ddaad1c6300'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:38.806576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1274/2590] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:38 2022] DELETE /identity/v3/users/0133a055a18347098a451ddaad1c6300 => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:38.811294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.811591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.811867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.812328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1317/2591] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:38 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:38.816948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.819166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:38.832849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:38.834091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:38.836050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be413e-ef4f-4f8d-8c79-59114be060ef admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:38.850237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=4,keystone:SELECT=86,keystone:DELETE=10 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:38.909293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e5ba2-3500-46ed-bfd5-7729e3f20db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:38.910268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e5ba2-3500-46ed-bfd5-7729e3f20db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.910496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e5ba2-3500-46ed-bfd5-7729e3f20db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.910774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e5ba2-3500-46ed-bfd5-7729e3f20db4 admin admin] PATH_INFO: `/v3/projects/0b7f6017de924b56962fa9751657dbfc/users/641ad7942b1649528b7103a582016dc8/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.912878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e5ba2-3500-46ed-bfd5-7729e3f20db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b7f6017de924b56962fa9751657dbfc, user_id=641ad7942b1649528b7103a582016dc8, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:38.916191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e5ba2-3500-46ed-bfd5-7729e3f20db4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:38.924535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1275/2592] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:38.928430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1318/2593] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:38 2022] PUT /identity/v3/projects/0b7f6017de924b56962fa9751657dbfc/users/641ad7942b1649528b7103a582016dc8/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:38.933849 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:38.935482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:38.943031 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:38.943327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:38.943605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:38.970012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:38.982911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] MFA Rules not processed for user `641ad7942b1649528b7103a582016dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:38.984516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:39.008736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8583a4-4ca4-48dc-9c7c-2941365840a5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:39.082880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:39.084029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:39.084537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:39.085106 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:39.095998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1319/2594] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:38 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:39.108021 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:39.109551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:39.178416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dcc4f6f-b08c-467a-a356-934daf235932 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:39.184142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:39.184919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:39.185216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:39.185498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:39.195815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:39.197683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:39.209559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:39.209975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1320/2595] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:39.212470 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:39.229811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1276/2596] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:39.428343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:39.440355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-415703ac-6a63-45d7-8f15-7f264a44f070 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:39.445621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:39.446237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:39.446526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:39.446971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] PATH_INFO: `/v3/users/010c328775884bb19625e1134c2e4277` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:39.448166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=010c328775884bb19625e1134c2e4277)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:39.450180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:39.689251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '010c328775884bb19625e1134c2e4277'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:39.689975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '010c328775884bb19625e1134c2e4277'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:39.690782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '010c328775884bb19625e1134c2e4277'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:39.707945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '010c328775884bb19625e1134c2e4277'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:39.717249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-3dba1113-b67a-4250-9cce-30779231bd01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '010c328775884bb19625e1134c2e4277'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:39.811853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:39.814257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:39.834915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1321/2597] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:39 2022] DELETE /identity/v3/users/010c328775884bb19625e1134c2e4277 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:39.840848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:39.847608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:39.847608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:39.847806 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-325981e2-1718-493b-8b50-a04a68d346ab service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:39.873228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:39.875358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:39.877192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:39.885949 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:39.924677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:39.926423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:39.927099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:39.927473 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] PATH_INFO: `/v3/projects/d9ba7f4c7bfc4dc59a176d00e0e0e7a4` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:39.929547 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d9ba7f4c7bfc4dc59a176d00e0e0e7a4)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:39.932970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:39.968395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1322/2598] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:39.980724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:39.983051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:40.062445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306f147-3a5f-4d85-a513-0a640ba4ac69 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:40.078453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd9ba7f4c7bfc4dc59a176d00e0e0e7a4', 'request_id': 'req-19b23b90-bd9e-4573-9c8c-0890f1724e00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.087510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd9ba7f4c7bfc4dc59a176d00e0e0e7a4', 'request_id': 'req-19b23b90-bd9e-4573-9c8c-0890f1724e00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.100744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd9ba7f4c7bfc4dc59a176d00e0e0e7a4', 'request_id': 'req-19b23b90-bd9e-4573-9c8c-0890f1724e00'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.131986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:40.132758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:40.133036 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:40.133313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:40.190633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] The token cache is being invalidate because project d9ba7f4c7bfc4dc59a176d00e0e0e7a4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:40.191759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.192448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.194154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1277/2599] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:39 2022] DELETE /identity/v3/projects/d9ba7f4c7bfc4dc59a176d00e0e0e7a4 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:40.198621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:40.200199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:40.203832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:40.205613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:40.215867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b23b90-bd9e-4573-9c8c-0890f1724e00 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:40.221370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1323/2600] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:40.221509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:40.222365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:40.222721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:40.223066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] PATH_INFO: `/v3/users/b4ef6515afdb4c1a8518d2324497eb0d` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:40.224381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4ef6515afdb4c1a8518d2324497eb0d)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:40.226407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:40.421100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4ef6515afdb4c1a8518d2324497eb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.435987 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4ef6515afdb4c1a8518d2324497eb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.458087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4ef6515afdb4c1a8518d2324497eb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.458901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4ef6515afdb4c1a8518d2324497eb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.527906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4ef6515afdb4c1a8518d2324497eb0d'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:40.535210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1278/2601] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:40 2022] DELETE /identity/v3/users/b4ef6515afdb4c1a8518d2324497eb0d => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:40.774150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:40.776720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:40.797472 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354cee02-2525-45e3-accc-37b6e880c7c1 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:40.836628 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=100 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:40.895229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:40.896834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:40.897411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:40.897915 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] PATH_INFO: `/v3/projects/3f8c11dd14894a3282a593ab7f7c9bf7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:40.901085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3f8c11dd14894a3282a593ab7f7c9bf7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:40.905050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:41.143264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3f8c11dd14894a3282a593ab7f7c9bf7', 'request_id': 'req-5290224f-8781-4800-891e-ba81f9b5c5ca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.151963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3f8c11dd14894a3282a593ab7f7c9bf7', 'request_id': 'req-5290224f-8781-4800-891e-ba81f9b5c5ca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.159254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3f8c11dd14894a3282a593ab7f7c9bf7', 'request_id': 'req-5290224f-8781-4800-891e-ba81f9b5c5ca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.165111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:41.169812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:41.195693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84c601a0-6a2d-44e7-9d3c-364dc31fb879 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:41.235948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:41.291111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:41.292001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:41.292283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:41.292557 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] PATH_INFO: `/v3/projects/b1936cebe4264746881ec84f0c67094f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:41.293953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b1936cebe4264746881ec84f0c67094f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:41.295893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:41.408214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] The token cache is being invalidate because project 3f8c11dd14894a3282a593ab7f7c9bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:41.408456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.409141 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.410777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1324/2602] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:40 2022] DELETE /identity/v3/projects/3f8c11dd14894a3282a593ab7f7c9bf7 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:41.415035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:41.417094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:41.430126 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5290224f-8781-4800-891e-ba81f9b5c5ca admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:41.435636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:41.436620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:41.436891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:41.437164 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] PATH_INFO: `/v3/users/641ad7942b1649528b7103a582016dc8` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:41.438390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=641ad7942b1649528b7103a582016dc8)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:41.440338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:41.465930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b1936cebe4264746881ec84f0c67094f', 'request_id': 'req-7685766e-0830-4be3-bfd9-80059651595b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.473520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b1936cebe4264746881ec84f0c67094f', 'request_id': 'req-7685766e-0830-4be3-bfd9-80059651595b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.480399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b1936cebe4264746881ec84f0c67094f', 'request_id': 'req-7685766e-0830-4be3-bfd9-80059651595b'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.595858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] The token cache is being invalidate because project b1936cebe4264746881ec84f0c67094f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:41.596150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.596880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7685766e-0830-4be3-bfd9-80059651595b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.598478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1279/2603] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:41 2022] DELETE /identity/v3/projects/b1936cebe4264746881ec84f0c67094f => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:41.609078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741b432-811a-42c2-ae94-04133d237570 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:41.609368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741b432-811a-42c2-ae94-04133d237570 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:41.609652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4741b432-811a-42c2-ae94-04133d237570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:41.630568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4741b432-811a-42c2-ae94-04133d237570 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:41.631812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4741b432-811a-42c2-ae94-04133d237570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:41.688242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '641ad7942b1649528b7103a582016dc8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.688996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '641ad7942b1649528b7103a582016dc8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.689642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '641ad7942b1649528b7103a582016dc8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.705564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '641ad7942b1649528b7103a582016dc8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.720364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '641ad7942b1649528b7103a582016dc8'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:41.720680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1280/2604] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:41.730576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:41.730865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:41.731150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:41.750622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:41.751884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:41.761134 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1281/2605] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:41.765824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:41.776725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a277af3-2c7d-463d-9502-e0066bb2aa2e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:41.781917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:41.782603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:41.782895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:41.783174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:41.784053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:41.785500 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:41.786218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:41.801017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1282/2606] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 14:11:41.810313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:41.821091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342bed6c-31ef-4188-a822-2d2c068ed1cc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:41.826960 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:41.827659 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:41.828474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:41.829215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:41.831401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:41.833395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:41.888505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1325/2607] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:41 2022] DELETE /identity/v3/users/641ad7942b1649528b7103a582016dc8 => generated 0 bytes in 474 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:41.966940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1283/2608] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:41 2022] POST /identity/v3/projects => generated 383 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:41.971665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.000034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:42.049014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778fe05a-65de-4b7e-9134-e92216fb96b6 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.115094 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364886f1-c1de-424d-9258-a599369876af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.115957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364886f1-c1de-424d-9258-a599369876af admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.116327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364886f1-c1de-424d-9258-a599369876af admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.116750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364886f1-c1de-424d-9258-a599369876af admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.117965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364886f1-c1de-424d-9258-a599369876af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.120290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364886f1-c1de-424d-9258-a599369876af admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.237765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1326/2609] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:41 2022] POST /identity/v3/users => generated 292 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:42.239638 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.252637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-480694dd-c0ab-4422-afd5-b9b289c89d7d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.258692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.260230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.260945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.261226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.261971 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.263733 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.277563 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1284/2610] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:42.283705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364886f1-c1de-424d-9258-a599369876af admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.296932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364886f1-c1de-424d-9258-a599369876af admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.302943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.303786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.304160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.304407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.305195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.307747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.320493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1327/2611] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:42 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:42.329905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.335399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:42.341817 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ece1527-3b28-40bd-9ade-43a012e565b6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.346596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e48b81-a329-48d1-a107-256e64f8dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.347509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48b81-a329-48d1-a107-256e64f8dbc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.347796 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48b81-a329-48d1-a107-256e64f8dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.349218 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e48b81-a329-48d1-a107-256e64f8dbc2 admin admin] PATH_INFO: `/v3/projects/fdd872e814c04aec9b794442435ad6e7/users/7c223a495fee4072b5d66f30ad2d4745/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.370487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e48b81-a329-48d1-a107-256e64f8dbc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd872e814c04aec9b794442435ad6e7, user_id=7c223a495fee4072b5d66f30ad2d4745, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.375595 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e48b81-a329-48d1-a107-256e64f8dbc2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.404465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.406490 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:42.420202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb1c9af-6cfb-4bdf-80bd-d12fb3e80d1a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.425588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.429685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.429685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.429685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] PATH_INFO: `/v3/projects/0b7f6017de924b56962fa9751657dbfc` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.429685 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b7f6017de924b56962fa9751657dbfc)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.430503 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.500294 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1285/2612] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:42 2022] PUT /identity/v3/projects/fdd872e814c04aec9b794442435ad6e7/users/7c223a495fee4072b5d66f30ad2d4745/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:42.511430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.511707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.511978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.550277 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] MFA Rules not processed for user `7c223a495fee4072b5d66f30ad2d4745`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:42.555751 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:42.573836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b7f6017de924b56962fa9751657dbfc', 'request_id': 'req-fa30a141-98fb-490b-a65a-5199d53b53c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:42.584441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b7f6017de924b56962fa9751657dbfc', 'request_id': 'req-fa30a141-98fb-490b-a65a-5199d53b53c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:42.593228 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b7f6017de924b56962fa9751657dbfc', 'request_id': 'req-fa30a141-98fb-490b-a65a-5199d53b53c5'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:42.642976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1286/2613] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:42.648279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] The token cache is being invalidate because project 0b7f6017de924b56962fa9751657dbfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:42.648760 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:42.648942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.649705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-fa30a141-98fb-490b-a65a-5199d53b53c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:42.652289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1328/2614] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:42 2022] DELETE /identity/v3/projects/0b7f6017de924b56962fa9751657dbfc => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:42.664669 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec382-cb55-4329-92d9-764a5c600546 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.665119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec382-cb55-4329-92d9-764a5c600546 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.666338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eec382-cb55-4329-92d9-764a5c600546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.691970 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c2eec382-cb55-4329-92d9-764a5c600546 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:42.693270 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c2eec382-cb55-4329-92d9-764a5c600546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:42.729470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:42.738554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2bd0f-9d2c-4a7f-a3e6-28f5f39fe20a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.743445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.744419 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.750090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.750090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.750090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.750090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.778676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1329/2615] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:42.785256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0317f-8008-4888-aece-0824e01d022c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.785554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0317f-8008-4888-aece-0824e01d022c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.785831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e0317f-8008-4888-aece-0824e01d022c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.806092 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-70e0317f-8008-4888-aece-0824e01d022c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:42.807447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-70e0317f-8008-4888-aece-0824e01d022c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:42.816045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1330/2616] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:42.820599 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0317f-8008-4888-aece-0824e01d022c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.831930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e0317f-8008-4888-aece-0824e01d022c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.839420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.840396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.840727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.841088 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.842321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:42.849412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.849559 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1287/2617] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:42 2022] POST /identity/v3/projects => generated 385 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:42.849659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.850546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.852333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:42.861872 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1331/2618] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 14:11:42.866506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:42.879139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb009ff-c73b-44ca-b2bb-33e70c11d56c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.911325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:42.927693 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e91a4a2-03f0-4aba-a41a-c57237e30993 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:42.933940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.934636 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.934919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.935197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.935914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.937993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:42.959813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:42.960493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:42.960821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:42.961093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:42.961817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:42.964808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.085103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1288/2619] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:42 2022] POST /identity/v3/users => generated 299 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:43.089604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1332/2620] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:42 2022] POST /identity/v3/projects => generated 373 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:43.090019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.099143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.101869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7beb3caa-95b7-474d-ba2e-aba5347df85f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.167955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.168706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.168997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.169279 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.170024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.171788 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.184272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1289/2621] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:43.188903 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77404232-67c3-4b5f-9486-9f54ae536408 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.189079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.193980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.194681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.194964 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.195240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.195945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.198658 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.201291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f5c2ff-d9b5-4cfe-8e96-9cac038c7b9d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.206629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.207616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.207902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.208182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] PATH_INFO: `/v3/projects/21ddfaea6e5342e7b0e0c7f50699faeb/users/40bd2929df41416fb872dfaa1fbd7481/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.225297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ddfaea6e5342e7b0e0c7f50699faeb, user_id=40bd2929df41416fb872dfaa1fbd7481, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.228124 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.340728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1333/2622] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:43 2022] POST /identity/v3/users => generated 287 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:43.344053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.361871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1290/2623] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:43 2022] PUT /identity/v3/projects/21ddfaea6e5342e7b0e0c7f50699faeb/users/40bd2929df41416fb872dfaa1fbd7481/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:43.362960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d4dc90-f2a7-4d23-b9f9-1225fc836731 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.363746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.403437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:43.455501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.456892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.457380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.458199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.459919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.463505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.469167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27df476-af47-416d-9a7a-ae3259e6d439 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.475994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.477194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.477671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.478714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.479722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1334/2624] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:43.480024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.482250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.484008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.495605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f8775e-0eb7-44b8-b171-fb520c37db26 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.495954 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1291/2625] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:43.501392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.501572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.502136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.502428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.502750 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.503498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.505177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.518243 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1335/2626] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:43 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:43.521047 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99e2876-69b0-4e28-82d8-e66fd5c80018 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.522706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.531460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee169ee7-e735-4445-a311-602d9625c026 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.533151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee169ee7-e735-4445-a311-602d9625c026 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.533713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee169ee7-e735-4445-a311-602d9625c026 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.534219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee169ee7-e735-4445-a311-602d9625c026 admin admin] PATH_INFO: `/v3/projects/21ddfaea6e5342e7b0e0c7f50699faeb/users/40bd2929df41416fb872dfaa1fbd7481/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.536554 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d39070-e220-4723-bba2-5fb36bd49955 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.538072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee169ee7-e735-4445-a311-602d9625c026 admin admin] RBAC: Authorizing `identity:create_grant(project_id=21ddfaea6e5342e7b0e0c7f50699faeb, user_id=40bd2929df41416fb872dfaa1fbd7481, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.541975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceedeedf-23c5-4946-87c3-c44e63492237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.543058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedeedf-23c5-4946-87c3-c44e63492237 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.543342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedeedf-23c5-4946-87c3-c44e63492237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.543491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee169ee7-e735-4445-a311-602d9625c026 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.543624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceedeedf-23c5-4946-87c3-c44e63492237 admin admin] PATH_INFO: `/v3/projects/0eef816416b9451986557c3c4172d24b/users/720ec4d2c4014e49b102f72710c1b597/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.564955 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1292/2627] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:43 2022] PUT /identity/v3/projects/21ddfaea6e5342e7b0e0c7f50699faeb/users/40bd2929df41416fb872dfaa1fbd7481/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:43.571272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedeedf-23c5-4946-87c3-c44e63492237 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0eef816416b9451986557c3c4172d24b, user_id=720ec4d2c4014e49b102f72710c1b597, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.577566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceedeedf-23c5-4946-87c3-c44e63492237 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.583342 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.583623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.584609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.620787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] MFA Rules not processed for user `40bd2929df41416fb872dfaa1fbd7481`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:43.622034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:43.661286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:43.690804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1336/2628] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:43 2022] PUT /identity/v3/projects/0eef816416b9451986557c3c4172d24b/users/720ec4d2c4014e49b102f72710c1b597/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:43.706890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.707376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.707840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.720407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1293/2629] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:43 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:43.737380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.739663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:43.741149 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] MFA Rules not processed for user `720ec4d2c4014e49b102f72710c1b597`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:43.742499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:43.807938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b02f78-f5ca-45bb-aa24-098ecd8e855a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.822293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1337/2630] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:43.827843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.865680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.866356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.866647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.866924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.922235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09105f74-1392-4537-8d78-f5190d4aa7cd None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:43.929234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:43.930322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:43.930799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:43.932577 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:43.933956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.937090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.941157 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:43.942725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:43.954366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1294/2631] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:43 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:43.984798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:43.996372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54241caa-fb8c-4bc8-956b-205b874f2f5f service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.001055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.001948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.002242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.004898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] PATH_INFO: `/v3/users/7c223a495fee4072b5d66f30ad2d4745` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.004898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c223a495fee4072b5d66f30ad2d4745)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.007024 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.031448 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:44.033268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:44.040122 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1338/2632] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:43 2022] POST /identity/v3/projects => generated 369 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:44.044735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.120917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb74a13-a08c-4caf-85a0-3eb789288752 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.125936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.126556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.126849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.127127 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.127871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.129881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.267710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c223a495fee4072b5d66f30ad2d4745'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:44.276476 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c223a495fee4072b5d66f30ad2d4745'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:44.294609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c223a495fee4072b5d66f30ad2d4745'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:44.295433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c223a495fee4072b5d66f30ad2d4745'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:44.333789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1339/2633] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:44 2022] POST /identity/v3/users => generated 291 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:44.338437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.348417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf47f7e9-2274-49e7-af93-3e86a8b6219f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.354409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.355073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.355365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.355647 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.356382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.358019 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.368188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1340/2634] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:44.372979 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.382897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9725e0e3-4cfc-4b12-a581-143e081c762c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.388176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.389170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.389457 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.389738 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] PATH_INFO: `/v3/projects/d23deb7e76ea4302817ae1a7a6d3a60f/users/bf67f13e4b56405c921a5d98c4bbd46a/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.405648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23deb7e76ea4302817ae1a7a6d3a60f, user_id=bf67f13e4b56405c921a5d98c4bbd46a, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.408300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.423555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1a0d7103-f65c-4fbb-bcb2-458e1aa90c03 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c223a495fee4072b5d66f30ad2d4745'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:44.430350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1295/2635] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:43 2022] DELETE /identity/v3/users/7c223a495fee4072b5d66f30ad2d4745 => generated 0 bytes in 446 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:44.436348 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.436618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.436913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.455680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:44.456979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:44.488231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1341/2636] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:44 2022] PUT /identity/v3/projects/d23deb7e76ea4302817ae1a7a6d3a60f/users/bf67f13e4b56405c921a5d98c4bbd46a/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:44.492404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.494676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:44.519060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1296/2637] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:44.531018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.532499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:44.570091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ebcdeef-6715-41d3-bf08-86c90aaa8d23 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.576528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.577244 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.577962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.578252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.578999 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.581843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.594693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1342/2638] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:44 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:44.598118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.598606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364cd9b3-5b57-48d7-a36e-10c7850cd721 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.608673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8617a2-2228-4f79-b367-9a6f93d8ae58 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.613523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbeb0cf-e912-47a8-978a-9edd42c8fa44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.614376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeb0cf-e912-47a8-978a-9edd42c8fa44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.614664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeb0cf-e912-47a8-978a-9edd42c8fa44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.614945 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbeb0cf-e912-47a8-978a-9edd42c8fa44 admin admin] PATH_INFO: `/v3/projects/d23deb7e76ea4302817ae1a7a6d3a60f/users/bf67f13e4b56405c921a5d98c4bbd46a/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.617052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeb0cf-e912-47a8-978a-9edd42c8fa44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d23deb7e76ea4302817ae1a7a6d3a60f, user_id=bf67f13e4b56405c921a5d98c4bbd46a, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.620024 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbeb0cf-e912-47a8-978a-9edd42c8fa44 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.632123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1343/2639] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:44 2022] PUT /identity/v3/projects/d23deb7e76ea4302817ae1a7a6d3a60f/users/bf67f13e4b56405c921a5d98c4bbd46a/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:44.644736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.645035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.645322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.656497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.657228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.657542 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.657820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.670215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.670215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.675162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] MFA Rules not processed for user `bf67f13e4b56405c921a5d98c4bbd46a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:44.676405 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:44.677352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:44.683164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1297/2640] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:44.711367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:44.766235 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1344/2641] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:44 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:44.779941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:44.781439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:44.838940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d956c048-77f9-4e89-aa46-079bb7b793c6 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:44.890694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:44.892840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:44.893089 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:44.893443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:44.904267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:44.905766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:44.917530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1298/2642] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:44 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:45.210056 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:45.211617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:45.289403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d064e8d2-45f2-4522-86f1-024559cb346b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:45.294672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:45.295545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:45.295845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:45.296123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] PATH_INFO: `/v3/projects/fdd872e814c04aec9b794442435ad6e7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:45.297345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd872e814c04aec9b794442435ad6e7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:45.299266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:45.338017 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:45.344335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b3c10-4f40-4173-a49e-11f1e89d8b4a service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:45.348048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:45.348863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:45.349151 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:45.349426 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:45.351241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:45.418452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:45.419894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:45.428972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1299/2643] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:45 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:45.460900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd872e814c04aec9b794442435ad6e7', 'request_id': 'req-a46c260d-2277-423f-9d01-dcfeff41d228'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.470089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd872e814c04aec9b794442435ad6e7', 'request_id': 'req-a46c260d-2277-423f-9d01-dcfeff41d228'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.476278 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd872e814c04aec9b794442435ad6e7', 'request_id': 'req-a46c260d-2277-423f-9d01-dcfeff41d228'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.567301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] The token cache is being invalidate because project fdd872e814c04aec9b794442435ad6e7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:45.567587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.568204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.569773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1345/2644] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:45 2022] DELETE /identity/v3/projects/fdd872e814c04aec9b794442435ad6e7 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:45.573963 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:45.586539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd5111a-c5d3-4329-887f-7c989cac1d8c service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:45.647092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:45.654010 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:45.654935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:45.655223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:45.655510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] PATH_INFO: `/v3/users/40bd2929df41416fb872dfaa1fbd7481` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:45.656769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40bd2929df41416fb872dfaa1fbd7481)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:45.658793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:45.940191 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40bd2929df41416fb872dfaa1fbd7481'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.953280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40bd2929df41416fb872dfaa1fbd7481'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.979951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40bd2929df41416fb872dfaa1fbd7481'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:45.980866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40bd2929df41416fb872dfaa1fbd7481'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.110216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-8a4a5531-ca30-4101-a78b-07ed556b70d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40bd2929df41416fb872dfaa1fbd7481'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.117092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1300/2645] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:45 2022] DELETE /identity/v3/users/40bd2929df41416fb872dfaa1fbd7481 => generated 0 bytes in 544 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:46.581917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:46.583443 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:46.597552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46c260d-2277-423f-9d01-dcfeff41d228 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:46.652951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:46.662332 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:46.663254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:46.663539 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:46.663882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] PATH_INFO: `/v3/projects/21ddfaea6e5342e7b0e0c7f50699faeb` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:46.665057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] RBAC: Authorizing `identity:delete_project(project_id=21ddfaea6e5342e7b0e0c7f50699faeb)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:46.667874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:46.802753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '21ddfaea6e5342e7b0e0c7f50699faeb', 'request_id': 'req-4062d098-7fe1-481a-9b22-01296dad4e30'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.811459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '21ddfaea6e5342e7b0e0c7f50699faeb', 'request_id': 'req-4062d098-7fe1-481a-9b22-01296dad4e30'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.818014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '21ddfaea6e5342e7b0e0c7f50699faeb', 'request_id': 'req-4062d098-7fe1-481a-9b22-01296dad4e30'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.938409 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] The token cache is being invalidate because project 21ddfaea6e5342e7b0e0c7f50699faeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:46.938702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.939369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4062d098-7fe1-481a-9b22-01296dad4e30 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:46.940980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1346/2646] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:46 2022] DELETE /identity/v3/projects/21ddfaea6e5342e7b0e0c7f50699faeb => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:46.946046 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:46.946562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:46.947054 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:46.966605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:46.967976 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:47.045452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1301/2647] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:47.047972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.048273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.048556 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.068530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:47.069817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:47.079368 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1347/2648] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:47.083941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.096776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426ca489-bce9-4c82-9e74-0e547d568a73 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:47.105783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:47.106952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.107527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.108012 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.109468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:47.110020 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:47.112254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:47.129303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1302/2649] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:11:47.135397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.145720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d1ad22a-659b-4e61-a977-d04d1a1cb758 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:47.156039 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:47.156866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.157230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.157606 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.158434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:47.161935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:47.314245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1348/2650] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:47 2022] POST /identity/v3/projects => generated 385 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:47.318867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.330552 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50633d60-6891-4c51-9296-772344b5c143 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:47.396431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:47.397158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.397445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.397721 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.398424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:47.401030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:47.410859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:47.512661 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1303/2651] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:47 2022] POST /identity/v3/users => generated 293 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:47.517993 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.529683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7888adb1-8609-4101-be17-1a9a36e84b7f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:47.535634 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:47.536484 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.536878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.537239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.538110 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:47.539976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:47.554488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1349/2652] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:47.559486 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.572956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3c91c8-3a34-4a45-b917-7b77b5c9da8d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:47.578253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37900dfc-9b54-4495-b0d6-b80fa7458421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:47.578957 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37900dfc-9b54-4495-b0d6-b80fa7458421 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.579245 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37900dfc-9b54-4495-b0d6-b80fa7458421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.579795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37900dfc-9b54-4495-b0d6-b80fa7458421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.580676 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37900dfc-9b54-4495-b0d6-b80fa7458421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:47.582324 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37900dfc-9b54-4495-b0d6-b80fa7458421 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:47.595809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1304/2653] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:11:47.600560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.618500 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610a5e3-ae4d-4cdd-89cc-cbbf34a0444c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:47.626677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:47.628205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.628741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.629214 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] PATH_INFO: `/v3/projects/02cb4ac018ea41828435f7c0f0782ee7/users/13a76963c56543a0a0be7219613d4271/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.654119 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02cb4ac018ea41828435f7c0f0782ee7, user_id=13a76963c56543a0a0be7219613d4271, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:47.659052 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:47.762614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1350/2654] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:47 2022] PUT /identity/v3/projects/02cb4ac018ea41828435f7c0f0782ee7/users/13a76963c56543a0a0be7219613d4271/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:47.774200 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:47.774590 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:47.775609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:47.807606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] MFA Rules not processed for user `13a76963c56543a0a0be7219613d4271`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:47.808945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:47.927516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1305/2655] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:47.945737 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:47.948679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:48.029533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23529514-29a2-4dfb-b788-46b8a9d71034 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:48.033589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e9fa9-f757-453b-b18c-df38d9261f30 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:48.034279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e9fa9-f757-453b-b18c-df38d9261f30 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:48.034565 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e9fa9-f757-453b-b18c-df38d9261f30 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:48.034858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e9fa9-f757-453b-b18c-df38d9261f30 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:48.045995 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e9fa9-f757-453b-b18c-df38d9261f30 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:48.047742 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e9fa9-f757-453b-b18c-df38d9261f30 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:48.058615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1351/2656] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:47 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:48.081783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:48.158183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53beec17-adbf-4107-920f-65f41ef49eb5 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:48.162996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:48.163878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:48.164160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:48.164438 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] PATH_INFO: `/v3/users/13a76963c56543a0a0be7219613d4271` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:48.165784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13a76963c56543a0a0be7219613d4271)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:48.167741 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:48.439745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13a76963c56543a0a0be7219613d4271'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:48.447205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13a76963c56543a0a0be7219613d4271'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:48.461146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13a76963c56543a0a0be7219613d4271'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:48.461903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13a76963c56543a0a0be7219613d4271'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:48.465137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=1,keystone:SELECT=90,keystone:DELETE=9 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:48.607689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13a76963c56543a0a0be7219613d4271'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:48.615214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1306/2657] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:11:48 2022] DELETE /identity/v3/users/13a76963c56543a0a0be7219613d4271 => generated 0 bytes in 535 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:48.621003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:48.621724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:48.621724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:48.641268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:48.642513 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:48.663751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:48.712957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1352/2658] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:48.723704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:48.725999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:48.792775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a35f18-28d5-4379-af94-92453debadd2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:48.796420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74be2e24-98f8-495e-8e49-78bdfe1482f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:48.797112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be2e24-98f8-495e-8e49-78bdfe1482f2 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:48.797395 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be2e24-98f8-495e-8e49-78bdfe1482f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:48.797664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74be2e24-98f8-495e-8e49-78bdfe1482f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:48.808189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be2e24-98f8-495e-8e49-78bdfe1482f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:48.809692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74be2e24-98f8-495e-8e49-78bdfe1482f2 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:48.820602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1307/2659] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:49.331018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:49.332833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:49.355337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506d56ce-a343-4c78-a252-5cc2381954e1 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:49.360997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:49.361901 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:49.362185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:49.362463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] PATH_INFO: `/v3/projects/02cb4ac018ea41828435f7c0f0782ee7` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:49.363671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02cb4ac018ea41828435f7c0f0782ee7)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:49.365642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:49.505827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02cb4ac018ea41828435f7c0f0782ee7', 'request_id': 'req-0e363e8d-6bc3-44f3-8020-52baf118d033'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:49.512891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02cb4ac018ea41828435f7c0f0782ee7', 'request_id': 'req-0e363e8d-6bc3-44f3-8020-52baf118d033'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:49.521481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02cb4ac018ea41828435f7c0f0782ee7', 'request_id': 'req-0e363e8d-6bc3-44f3-8020-52baf118d033'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:49.607335 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] The token cache is being invalidate because project 02cb4ac018ea41828435f7c0f0782ee7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:11:49.607640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:49.608469 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-0e363e8d-6bc3-44f3-8020-52baf118d033 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:11:49.609988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1353/2660] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:11:49 2022] DELETE /identity/v3/projects/02cb4ac018ea41828435f7c0f0782ee7 => generated 0 bytes in 280 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:49.615287 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:49.615767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:49.616008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:49.635228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:49.636435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:49.703216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1308/2661] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:49.766781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:49.767327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:49.767327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:49.787508 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:49.788766 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:49.797321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1354/2662] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:11:49.801700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:49.812942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd69b987-941e-4b62-aee5-7f0f3cf5bc53 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:49.817570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:49.818234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:49.818524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:49.818807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:49.819541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:11:49.819854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:49.821541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:49.834186 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1309/2663] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:11:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:11:49.839063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:49.848280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f1ba27-37d0-43f0-9749-d137be39e079 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:49.853716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:49.854369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:49.854708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:49.854985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:49.855683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:49.857676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:49.981908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1355/2664] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:49 2022] POST /identity/v3/projects => generated 385 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:49.986223 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:49.997403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824e62bf-8732-4ff6-86ab-86fd274345ed admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.060464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.061183 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.061472 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.061757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.062458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.064406 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.157475 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1310/2665] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:49 2022] POST /identity/v3/users => generated 293 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:50.161453 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.172982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df43c662-54d1-4e7a-b733-7b56d83514d2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.177932 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.183786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.183786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.183786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.183786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.183786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.193113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1356/2666] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:50.197939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.211487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73801dbd-aaab-4d29-a04e-2be7e819fe79 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.217205 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e86d78c-34c6-4557-8974-132413f9f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.217885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d78c-34c6-4557-8974-132413f9f1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.218176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d78c-34c6-4557-8974-132413f9f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.218460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e86d78c-34c6-4557-8974-132413f9f1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.219201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86d78c-34c6-4557-8974-132413f9f1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.220923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e86d78c-34c6-4557-8974-132413f9f1ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.227504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:50.234915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1311/2667] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:50.239419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.250638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c05084-7f57-4106-b87f-db3e25c85b90 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.255981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.256956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.261182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.261182 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] PATH_INFO: `/v3/projects/173d0366f2b040f1b95f889326e27fea/users/3a7f8dceaac849edb4574b68b1897d0b/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.277630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=173d0366f2b040f1b95f889326e27fea, user_id=3a7f8dceaac849edb4574b68b1897d0b, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.280413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.352580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1357/2668] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:50 2022] PUT /identity/v3/projects/173d0366f2b040f1b95f889326e27fea/users/3a7f8dceaac849edb4574b68b1897d0b/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:50.363259 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.363546 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.363824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.392706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] MFA Rules not processed for user `3a7f8dceaac849edb4574b68b1897d0b`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:50.394606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:50.464023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1312/2669] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:50 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:50.468131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.515616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:50.534788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba98940-8881-4895-8f75-27179569bb7f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.539593 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.540253 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.540711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.541216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.542007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.543925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.635133 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1358/2670] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:50 2022] POST /identity/v3/projects => generated 383 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:50.636646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.648149 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7182f0f7-6784-4983-9d0e-cf152c9ec8e9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.736264 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.741474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.741474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.741474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.741474 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.742469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.836443 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1313/2671] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:50 2022] POST /identity/v3/users => generated 292 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:50.841076 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.851835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a804db4-c3a7-4899-b56f-a642467fb80d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.856667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.861580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.861580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.861580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.861580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.861580 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.871727 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1359/2672] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:50.875918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.887105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded45a84-dd91-49ea-86fd-ab0f1d73459d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.891990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81424c44-0cc4-4103-ad9f-1a7e60a8e25a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.892694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81424c44-0cc4-4103-ad9f-1a7e60a8e25a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.892986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81424c44-0cc4-4103-ad9f-1a7e60a8e25a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.893269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81424c44-0cc4-4103-ad9f-1a7e60a8e25a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.893998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81424c44-0cc4-4103-ad9f-1a7e60a8e25a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.895613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81424c44-0cc4-4103-ad9f-1a7e60a8e25a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:50.906449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1314/2673] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:50.910891 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:50.921530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51daeaf-1b3e-40da-ac65-cec07581c889 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:50.926186 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:50.927073 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:50.927367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:50.927648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] PATH_INFO: `/v3/projects/9edf458a5d5b41078cc088c75cc00f92/users/a18c655607334be9905bc499eeaa2816/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:50.943477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9edf458a5d5b41078cc088c75cc00f92, user_id=a18c655607334be9905bc499eeaa2816, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:50.946197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.036202 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1360/2674] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:50 2022] PUT /identity/v3/projects/9edf458a5d5b41078cc088c75cc00f92/users/a18c655607334be9905bc499eeaa2816/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:51.046966 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.047263 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.047550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.089916 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] MFA Rules not processed for user `a18c655607334be9905bc499eeaa2816`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:51.091166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:51.154653 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:51.159798 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1315/2675] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:51.169460 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.236833 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90169a1-0037-44d2-9b90-bc4dacf409a4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:51.241579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:51.242241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.242529 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.242877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.243531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:51.246299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.370280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1361/2676] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:11:51 2022] POST /identity/v3/projects => generated 385 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:11:51.374736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.386002 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5999c46e-8478-4fef-9873-56297417fe52 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:51.461132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:51.461891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.462175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.462983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.463679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:51.465737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.568811 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1316/2677] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:11:51 2022] POST /identity/v3/users => generated 299 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:11:51.573432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.584656 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc769429-05f4-4118-9f10-c1e097ae59c3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:51.590053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:51.590717 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.591020 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.591269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.591989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:51.593720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.605317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1362/2678] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:51.609993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.625606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f907a676-351f-41ba-86cb-c93f881403c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:51.626230 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:51.627116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.627401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.627680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] PATH_INFO: `/v3/projects/cb68e00538d74236bee839ed26d6f7c0/users/307d73d3c5b14892b15791033af3d602/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.645343 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb68e00538d74236bee839ed26d6f7c0, user_id=307d73d3c5b14892b15791033af3d602, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:51.648378 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.710487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1317/2679] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:51 2022] PUT /identity/v3/projects/cb68e00538d74236bee839ed26d6f7c0/users/307d73d3c5b14892b15791033af3d602/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:11:51.715308 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.725268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5988f48-934a-460d-99df-6de6bc8ca4ff admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:51.792842 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a5e6ee-94eb-4f21-830b-3666524deebf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:51.793519 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5e6ee-94eb-4f21-830b-3666524deebf admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.793800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5e6ee-94eb-4f21-830b-3666524deebf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.794074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a5e6ee-94eb-4f21-830b-3666524deebf admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.794800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5e6ee-94eb-4f21-830b-3666524deebf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:51.796393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a5e6ee-94eb-4f21-830b-3666524deebf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.812390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1363/2680] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:11:51 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:11:51.816914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.828050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1e35c6-1b1b-4d4a-ad06-a448a120155d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:51.833064 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:51.833935 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.834213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.834483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] PATH_INFO: `/v3/projects/cb68e00538d74236bee839ed26d6f7c0/users/307d73d3c5b14892b15791033af3d602/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.836409 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb68e00538d74236bee839ed26d6f7c0, user_id=307d73d3c5b14892b15791033af3d602, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:51.839708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:51.850104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1318/2681] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:11:51 2022] PUT /identity/v3/projects/cb68e00538d74236bee839ed26d6f7c0/users/307d73d3c5b14892b15791033af3d602/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:11:51.861419 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:51.862066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:51.862066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:51.867881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:51.889707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] MFA Rules not processed for user `307d73d3c5b14892b15791033af3d602`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:11:51.890892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:51.962386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1364/2682] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:11:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:11:51.974499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:51.976276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:11:52.033871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b28743c3-f1c7-4e2f-96af-296a0a9c8338 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:52.037704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:52.038368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:52.038660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:52.038942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:52.096699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:52.098154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:52.107681 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1319/2683] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:51 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:11:55.286688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:55.296777 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195e914f-83f3-479b-b48b-b27a058bd528 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:55.300609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:55.301329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:55.301617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:55.301897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:55.366045 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:55.367506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:55.383632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1365/2684] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:55 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:11:57.410642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:11:57.414235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:11:57.417537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038b2679-0018-483c-86b0-2cc8fbbec762 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:11:57.422851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065cfac7-639e-4963-8642-abc141db6b92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:11:57.423503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cfac7-639e-4963-8642-abc141db6b92 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:11:57.423784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cfac7-639e-4963-8642-abc141db6b92 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:11:57.424060 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065cfac7-639e-4963-8642-abc141db6b92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:11:57.441175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cfac7-639e-4963-8642-abc141db6b92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:11:57.441175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065cfac7-639e-4963-8642-abc141db6b92 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:11:57.448744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1320/2685] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:11:57 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:12:03.837746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:03.914834 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d91b4f5-a854-4fdd-957b-950d4b3fe0e9 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:03.919571 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:03.921238 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:03.921997 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:03.922280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] PATH_INFO: `/v3/users/3a7f8dceaac849edb4574b68b1897d0b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:03.923512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a7f8dceaac849edb4574b68b1897d0b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:03.925497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:04.021108 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:04.131015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a7f8dceaac849edb4574b68b1897d0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:04.131683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a7f8dceaac849edb4574b68b1897d0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:04.132309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a7f8dceaac849edb4574b68b1897d0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:04.145307 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a7f8dceaac849edb4574b68b1897d0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:04.152385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a7f8dceaac849edb4574b68b1897d0b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:04.230659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1366/2686] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:12:03 2022] DELETE /identity/v3/users/3a7f8dceaac849edb4574b68b1897d0b => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:04.235611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:04.236193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:04.236193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:04.255103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:04.256374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:04.319084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1321/2687] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:04.330430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:04.331861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:04.390732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78010609-cba3-4031-b74a-88e9e81ded7c admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:04.394397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:04.395057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:04.395342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:04.395620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:04.405986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:04.407448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:04.417260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1367/2688] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:12:04.889527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:04.891687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:04.903760 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f96a19-38e0-492c-bf31-76e1701b08aa None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:04.909598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:04.910494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:04.910777 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:04.911177 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] PATH_INFO: `/v3/projects/173d0366f2b040f1b95f889326e27fea` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:04.912451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] RBAC: Authorizing `identity:delete_project(project_id=173d0366f2b040f1b95f889326e27fea)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:04.914534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:05.078897 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '173d0366f2b040f1b95f889326e27fea', 'request_id': 'req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.086111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '173d0366f2b040f1b95f889326e27fea', 'request_id': 'req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.093280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '173d0366f2b040f1b95f889326e27fea', 'request_id': 'req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.225745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] The token cache is being invalidate because project 173d0366f2b040f1b95f889326e27fea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:12:05.226071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.226912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.229085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1322/2689] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:12:04 2022] DELETE /identity/v3/projects/173d0366f2b040f1b95f889326e27fea => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:05.234485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:05.246100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25e2b37c-2746-4af4-a68c-f1e557d6a8ce service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:05.318783 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:05.319706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:05.319992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:05.320328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] PATH_INFO: `/v3/users/a18c655607334be9905bc499eeaa2816` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:05.321562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] RBAC: Authorizing `identity:delete_user(user_id=a18c655607334be9905bc499eeaa2816)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:05.323527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:05.640809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a18c655607334be9905bc499eeaa2816'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.641595 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a18c655607334be9905bc499eeaa2816'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.642369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a18c655607334be9905bc499eeaa2816'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.645291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:05.657722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a18c655607334be9905bc499eeaa2816'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.664706 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a18c655607334be9905bc499eeaa2816'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:05.756242 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1368/2690] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:12:05 2022] DELETE /identity/v3/users/a18c655607334be9905bc499eeaa2816 => generated 0 bytes in 523 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:06.347883 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:06.349832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:06.364959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6db80d12-8a2c-4512-a8a9-2251ccd6a42a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:06.430034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:06.430905 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:06.431188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:06.431464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] PATH_INFO: `/v3/projects/9edf458a5d5b41078cc088c75cc00f92` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:06.432675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9edf458a5d5b41078cc088c75cc00f92)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:06.434598 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:06.453981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:06.573918 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9edf458a5d5b41078cc088c75cc00f92', 'request_id': 'req-46fdf97c-c058-475a-b913-23f7ea859eb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:06.581482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9edf458a5d5b41078cc088c75cc00f92', 'request_id': 'req-46fdf97c-c058-475a-b913-23f7ea859eb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:06.589260 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9edf458a5d5b41078cc088c75cc00f92', 'request_id': 'req-46fdf97c-c058-475a-b913-23f7ea859eb6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:06.687683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] The token cache is being invalidate because project 9edf458a5d5b41078cc088c75cc00f92 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:12:06.687964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:06.688628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:06.690171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1323/2691] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:12:06 2022] DELETE /identity/v3/projects/9edf458a5d5b41078cc088c75cc00f92 => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:12:06.694305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:06.695712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:06.709102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b5a30e-c7d5-47bd-bdbe-a9beeaea644a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:06.769266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:06.770294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:06.770579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:06.770855 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] PATH_INFO: `/v3/users/307d73d3c5b14892b15791033af3d602` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:06.772212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=307d73d3c5b14892b15791033af3d602)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:06.774089 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:07.143874 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '307d73d3c5b14892b15791033af3d602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:07.144520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '307d73d3c5b14892b15791033af3d602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:07.145195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '307d73d3c5b14892b15791033af3d602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:07.158254 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '307d73d3c5b14892b15791033af3d602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:07.165212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ee2307f1-f50a-4c89-8567-7853a9255d43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '307d73d3c5b14892b15791033af3d602'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:07.331388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1369/2692] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:12:06 2022] DELETE /identity/v3/users/307d73d3c5b14892b15791033af3d602 => generated 0 bytes in 638 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:07.797737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:07.799226 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:07.814454 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fdf97c-c058-475a-b913-23f7ea859eb6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:07.882756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:07.883820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:07.884114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:07.884389 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] PATH_INFO: `/v3/projects/cb68e00538d74236bee839ed26d6f7c0` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:07.885974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb68e00538d74236bee839ed26d6f7c0)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:07.887864 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.048875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb68e00538d74236bee839ed26d6f7c0', 'request_id': 'req-fa87f4b8-b0a0-49d2-873e-6e8707a68875'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:08.055204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb68e00538d74236bee839ed26d6f7c0', 'request_id': 'req-fa87f4b8-b0a0-49d2-873e-6e8707a68875'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:08.061584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb68e00538d74236bee839ed26d6f7c0', 'request_id': 'req-fa87f4b8-b0a0-49d2-873e-6e8707a68875'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:08.155187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] The token cache is being invalidate because project cb68e00538d74236bee839ed26d6f7c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:12:08.155471 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:08.156194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fa87f4b8-b0a0-49d2-873e-6e8707a68875 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:08.157774 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1324/2693] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:12:07 2022] DELETE /identity/v3/projects/cb68e00538d74236bee839ed26d6f7c0 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:08.163926 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.164420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.164515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.183200 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:08.184423 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:08.229227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:08.247424 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1370/2694] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:08.255964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.256253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.256537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.275129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:08.276326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:08.284720 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1325/2695] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:08.289085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:08.302764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7ead2b-21da-4f41-99a3-1d08e05a3ec8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:08.304139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:08.304849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.305129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.305412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.306144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:12:08.306454 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:08.308051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.320719 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1371/2696] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:12:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:12:08.325156 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:08.335069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e9a47-f73f-4db3-a04a-34edf0d1af5d None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:08.339757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:08.340423 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.340743 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.341023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.341728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:08.343977 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.495697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1326/2697] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:12:08 2022] POST /identity/v3/projects => generated 377 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:08.502230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:08.512725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a45aa31-108c-47dd-8f47-419bbfdba1aa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:08.573640 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:08.574289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.574584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.574862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.575569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:08.578198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.677619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1372/2698] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:12:08 2022] POST /identity/v3/users => generated 289 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:08.682192 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:08.693804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938759d5-0f48-4eb8-9f56-b48520e17b85 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:08.699070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:08.699611 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.705413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.705413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.705413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:08.705413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.705413 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:08.715333 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1327/2699] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:08.720778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:08.731830 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2985c13-2858-4ce1-9af3-9c37191aa1d3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:08.737478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b08562-4899-4a95-9a7b-691308989a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:08.738170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08562-4899-4a95-9a7b-691308989a51 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.738459 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08562-4899-4a95-9a7b-691308989a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.738745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b08562-4899-4a95-9a7b-691308989a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.739497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b08562-4899-4a95-9a7b-691308989a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:08.741188 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b08562-4899-4a95-9a7b-691308989a51 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.753353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1373/2700] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:08.757901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:08.769261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa1866e-19c0-4e9e-ba01-1823c08426ea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:08.774085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:08.774986 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.775272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.775553 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] PATH_INFO: `/v3/projects/aee15038cdf945a594437e83a34e1afe/users/7b6534d42be64c3cb311d384458bfae3/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.791216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee15038cdf945a594437e83a34e1afe, user_id=7b6534d42be64c3cb311d384458bfae3, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:08.793974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:08.904261 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1328/2701] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:12:08 2022] PUT /identity/v3/projects/aee15038cdf945a594437e83a34e1afe/users/7b6534d42be64c3cb311d384458bfae3/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:08.910128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ebbdd-5581-4065-82c0-feb2d6a37d53 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:08.910432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ebbdd-5581-4065-82c0-feb2d6a37d53 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:08.910747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369ebbdd-5581-4065-82c0-feb2d6a37d53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:08.930393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-369ebbdd-5581-4065-82c0-feb2d6a37d53 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:08.931704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-369ebbdd-5581-4065-82c0-feb2d6a37d53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:08.995619 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1374/2702] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:09.005881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:09.007662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:09.068179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18929f30-fbb7-4253-9204-111c38371e0f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:09.074753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:09.074753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:09.074753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:09.074753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:09.084138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:09.085630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:09.096538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1329/2703] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:12:14.880017 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:14.880659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:14.880745 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:14.925369 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] MFA Rules not processed for user `7b6534d42be64c3cb311d384458bfae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:14.926708 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:14.980445 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:15.012868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1375/2704] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:14 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:15.018478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:15.029187 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41495eb1-3bd2-4da5-9ccc-28b72ec25982 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:15.038949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:15.038949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:15.038949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:15.038949 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:15.039433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:15.041837 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:15.169466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1330/2705] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:12:15 2022] POST /identity/v3/projects => generated 379 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:15.173873 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:15.186199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-969aee26-3dfa-4195-ac75-276d1ebdb94d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:15.250309 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:15.251006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:15.251301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:15.251583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:15.252296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:15.254305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:15.350906 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1376/2706] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:12:15 2022] POST /identity/v3/users => generated 296 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:15.354932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:15.368678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cb702-834e-4463-aea9-36b0ab8fb943 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:15.371902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:15.372609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:15.372900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:15.373178 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:15.374815 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:15.376266 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:15.392607 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1331/2707] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:15.396618 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:15.408765 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2711a2-4d4f-445f-bb5c-46840cc3052f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:15.413739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:15.414780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:15.415060 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:15.415333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] PATH_INFO: `/v3/projects/702a8f83e4db4e0296fcfadcedb6b5b1/users/3268c3cede5e4635ab3cb9390c407c0e/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:15.433249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702a8f83e4db4e0296fcfadcedb6b5b1, user_id=3268c3cede5e4635ab3cb9390c407c0e, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:15.436683 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:15.536828 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1377/2708] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:12:15 2022] PUT /identity/v3/projects/702a8f83e4db4e0296fcfadcedb6b5b1/users/3268c3cede5e4635ab3cb9390c407c0e/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:12:15.540602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:15.551705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781d854c-76e1-4173-bf35-96b5ddd48ed4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:15.610579 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907891a0-2d03-445c-be4d-22e1adf6a996 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:15.611256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907891a0-2d03-445c-be4d-22e1adf6a996 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:15.611540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907891a0-2d03-445c-be4d-22e1adf6a996 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:15.611812 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907891a0-2d03-445c-be4d-22e1adf6a996 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:15.612539 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907891a0-2d03-445c-be4d-22e1adf6a996 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:15.614179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907891a0-2d03-445c-be4d-22e1adf6a996 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:15.625217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1332/2709] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:15 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:15.629741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:15.641213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02da00a-42db-4c40-8492-5762e0e17c28 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:15.645831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:15.646721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:15.647004 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:15.647277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] PATH_INFO: `/v3/projects/702a8f83e4db4e0296fcfadcedb6b5b1/users/3268c3cede5e4635ab3cb9390c407c0e/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:15.649260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] RBAC: Authorizing `identity:create_grant(project_id=702a8f83e4db4e0296fcfadcedb6b5b1, user_id=3268c3cede5e4635ab3cb9390c407c0e, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:15.651867 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:15.663313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1378/2710] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:12:15 2022] PUT /identity/v3/projects/702a8f83e4db4e0296fcfadcedb6b5b1/users/3268c3cede5e4635ab3cb9390c407c0e/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:21.837516 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:21.838368 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:21.838658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:21.879396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] MFA Rules not processed for user `3268c3cede5e4635ab3cb9390c407c0e`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:21.880910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:21.923570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:21.975700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1333/2711] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:21 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:21.993356 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:21.995155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:22.083854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4629c370-44d8-42aa-8ef0-c5d798416c12 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:22.089615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d557ae94-6d07-4c17-b1b5-af33807f5266 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:22.090279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557ae94-6d07-4c17-b1b5-af33807f5266 service nova] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:22.090643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557ae94-6d07-4c17-b1b5-af33807f5266 service nova] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:22.090904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d557ae94-6d07-4c17-b1b5-af33807f5266 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:22.127080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:22.155839 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557ae94-6d07-4c17-b1b5-af33807f5266 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:22.158191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d557ae94-6d07-4c17-b1b5-af33807f5266 service nova] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:22.168054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1379/2712] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:12:32.024803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:32.184448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=13 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:53.321305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:53.322769 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:53.389153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7fdc6d-d177-44bf-8773-40a46f9d9e1b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:53.393972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:53.394859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:53.395150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:53.395429 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] PATH_INFO: `/v3/users/720ec4d2c4014e49b102f72710c1b597` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:53.396695 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=720ec4d2c4014e49b102f72710c1b597)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:53.398639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:53.616642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '720ec4d2c4014e49b102f72710c1b597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:53.629536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '720ec4d2c4014e49b102f72710c1b597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:53.651361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '720ec4d2c4014e49b102f72710c1b597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:53.652339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '720ec4d2c4014e49b102f72710c1b597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:53.728818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '720ec4d2c4014e49b102f72710c1b597'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:53.736420 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1334/2713] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:12:53 2022] DELETE /identity/v3/users/720ec4d2c4014e49b102f72710c1b597 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:53.741439 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:53.741913 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:53.742007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:53.759789 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:53.763303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:53.824224 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1380/2714] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:53.835754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:53.837222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:53.906446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c788b760-1daa-4298-b372-474f3f7811ca admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:53.910794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:53.911504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:53.911768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:53.912049 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:53.921982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:53.924027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:53.934680 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1335/2715] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:12:54.387083 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:54.388524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:54.401219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac0c27c2-b25b-4d75-a67d-492512504485 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:54.406038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:54.407051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:54.407501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:54.407836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] PATH_INFO: `/v3/projects/0eef816416b9451986557c3c4172d24b` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:54.409179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0eef816416b9451986557c3c4172d24b)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:54.411099 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:54.541617 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0eef816416b9451986557c3c4172d24b', 'request_id': 'req-aef647f5-b610-4bb0-83bc-ee884b4c6b29'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:54.549190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0eef816416b9451986557c3c4172d24b', 'request_id': 'req-aef647f5-b610-4bb0-83bc-ee884b4c6b29'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:54.555142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0eef816416b9451986557c3c4172d24b', 'request_id': 'req-aef647f5-b610-4bb0-83bc-ee884b4c6b29'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:54.648057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] The token cache is being invalidate because project 0eef816416b9451986557c3c4172d24b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:12:54.648347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:54.649010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:54.650558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1381/2716] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:12:54 2022] DELETE /identity/v3/projects/0eef816416b9451986557c3c4172d24b => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:54.654783 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:54.665321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94db682c-b362-40e2-9e00-fcb6be12e290 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:54.685292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:54.731911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:54.732855 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:54.733221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:54.733421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] PATH_INFO: `/v3/users/bf67f13e4b56405c921a5d98c4bbd46a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:54.734645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bf67f13e4b56405c921a5d98c4bbd46a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:54.738697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:55.015063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bf67f13e4b56405c921a5d98c4bbd46a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.024135 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bf67f13e4b56405c921a5d98c4bbd46a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.037214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bf67f13e4b56405c921a5d98c4bbd46a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.037945 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bf67f13e4b56405c921a5d98c4bbd46a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.156309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-067ee0c3-6612-42ac-b806-5fc9d76d16a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bf67f13e4b56405c921a5d98c4bbd46a'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.163756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1336/2717] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:12:54 2022] DELETE /identity/v3/users/bf67f13e4b56405c921a5d98c4bbd46a => generated 0 bytes in 510 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:12:55.588303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:55.589835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:55.603845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef647f5-b610-4bb0-83bc-ee884b4c6b29 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:55.678977 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:55.679856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:55.680165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:55.680442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] PATH_INFO: `/v3/projects/d23deb7e76ea4302817ae1a7a6d3a60f` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:55.681688 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d23deb7e76ea4302817ae1a7a6d3a60f)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:55.683612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:55.786908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:55.803542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd23deb7e76ea4302817ae1a7a6d3a60f', 'request_id': 'req-8feba4ba-5f3b-4feb-816a-3a728080636f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.809882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd23deb7e76ea4302817ae1a7a6d3a60f', 'request_id': 'req-8feba4ba-5f3b-4feb-816a-3a728080636f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.816117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd23deb7e76ea4302817ae1a7a6d3a60f', 'request_id': 'req-8feba4ba-5f3b-4feb-816a-3a728080636f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.923771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] The token cache is being invalidate because project d23deb7e76ea4302817ae1a7a6d3a60f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:12:55.924635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.925341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8feba4ba-5f3b-4feb-816a-3a728080636f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:12:55.927293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1382/2718] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:12:55 2022] DELETE /identity/v3/projects/d23deb7e76ea4302817ae1a7a6d3a60f => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:55.934067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:55.934600 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:55.934699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:55.954823 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:55.966792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:56.019862 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1337/2719] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:12:56.028718 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.029013 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.029294 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.048721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:56.050447 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:56.058489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1383/2720] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:56.062970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.074126 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9698827-7dcf-4a8a-bb64-48f00e568774 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.079086 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.079755 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.080036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.080614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.081401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:12:56.081705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.083328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.096520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1338/2721] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:12:56.100722 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.110512 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22195474-a5b1-4abc-9bac-b1caec730374 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.116007 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.116715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.117005 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.117283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.118082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.119957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.204937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1384/2722] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:12:56 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:56.210731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.222317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9480ff4-7c32-441e-9a00-ffa802817496 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.254959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:56.294285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.294973 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.295254 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.295535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.296235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.298210 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.400350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1339/2723] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:12:56 2022] POST /identity/v3/users => generated 297 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:56.406720 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.419151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542c37a-1b9f-4cc1-995d-cc24d6bf6d22 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.425809 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.428532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.429290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.429581 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.430344 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.431991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.452491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1385/2724] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:56.452702 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.461993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407e7a0f-647b-4f74-b226-84d56f362924 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.466846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44167c6-5561-456e-9ea4-3cb846082a95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.467505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44167c6-5561-456e-9ea4-3cb846082a95 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.467792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44167c6-5561-456e-9ea4-3cb846082a95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.468071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44167c6-5561-456e-9ea4-3cb846082a95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.468834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44167c6-5561-456e-9ea4-3cb846082a95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.470610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44167c6-5561-456e-9ea4-3cb846082a95 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.482120 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1340/2725] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:56.486300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.497576 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88bebd7-a323-4982-87fc-50960103aa73 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.503764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.505390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.505680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.505962 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] PATH_INFO: `/v3/projects/553b3bbdb8334ccca35f8c615c237994/users/99f138f0386c4c3d9c7a02e23da0fcaa/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.528302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=553b3bbdb8334ccca35f8c615c237994, user_id=99f138f0386c4c3d9c7a02e23da0fcaa, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.531068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.605450 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1386/2726] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:12:56 2022] PUT /identity/v3/projects/553b3bbdb8334ccca35f8c615c237994/users/99f138f0386c4c3d9c7a02e23da0fcaa/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:56.617437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.617852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.618280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.657886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] MFA Rules not processed for user `99f138f0386c4c3d9c7a02e23da0fcaa`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:56.659504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:56.686749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.688483 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:56.758965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1341/2727] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:56.762835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.765367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e08f07-d7c1-445b-9583-86dd419c35cc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.770918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.771626 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.772074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.772655 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.775041 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:56.858197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53f161d-ed9e-47c3-b4d8-feb5a879b722 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:56.864044 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:56.864744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:56.865023 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:56.865300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:56.865994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.867902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.876251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:56.880517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:56.882117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:56.911241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1387/2728] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:12:56.971964 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1342/2729] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:12:56 2022] POST /identity/v3/projects => generated 393 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:12:56.975965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:56.992644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-357af736-5ed9-4a97-9a2a-a96035b2e50e service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.071159 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.072097 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.072547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.072864 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.073560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.075520 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.209801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1388/2730] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:12:56 2022] POST /identity/v3/users => generated 303 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:12:57.214582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:57.230133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1f023b-ae30-407a-a6d0-8a6b2a0a08cd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.235163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.235856 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.236144 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.237037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.238236 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.239901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.257379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1343/2731] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:12:57.262085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:57.276951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8944413e-4d3d-4e2b-afa8-61db9c0bbdca admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.282296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.283616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.283616 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.283960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] PATH_INFO: `/v3/projects/b73a20391dd245cf8bda708eb7501160/users/ffa988145873427c983d5106e876d6dd/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.303878 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73a20391dd245cf8bda708eb7501160, user_id=ffa988145873427c983d5106e876d6dd, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.306687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.403192 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1389/2732] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:12:57 2022] PUT /identity/v3/projects/b73a20391dd245cf8bda708eb7501160/users/ffa988145873427c983d5106e876d6dd/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:57.407326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:57.418882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eb0b3c-d6b7-46d7-8f34-f5638ccce1bb admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.427885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:12:57.480943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4a8d8b-6f51-4b0a-85e6-fc2455f0ad0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.481618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a8d8b-6f51-4b0a-85e6-fc2455f0ad0a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.481900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a8d8b-6f51-4b0a-85e6-fc2455f0ad0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.482174 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4a8d8b-6f51-4b0a-85e6-fc2455f0ad0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.482898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a8d8b-6f51-4b0a-85e6-fc2455f0ad0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.484677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4a8d8b-6f51-4b0a-85e6-fc2455f0ad0a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.495308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1344/2733] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:12:57.499911 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:57.510648 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb59809-ffb5-4f3b-88d4-64b24df36a75 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.515824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.516768 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.517051 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.517328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] PATH_INFO: `/v3/projects/b73a20391dd245cf8bda708eb7501160/users/ffa988145873427c983d5106e876d6dd/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.519925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b73a20391dd245cf8bda708eb7501160, user_id=ffa988145873427c983d5106e876d6dd, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.522741 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.533841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1390/2734] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:12:57 2022] PUT /identity/v3/projects/b73a20391dd245cf8bda708eb7501160/users/ffa988145873427c983d5106e876d6dd/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:12:57.545633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.545930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.546215 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.586280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] MFA Rules not processed for user `ffa988145873427c983d5106e876d6dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:12:57.587531 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:57.660522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1345/2735] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:12:57 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:12:57.671613 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:57.673359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:12:57.732208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27c1b88-0c82-4bed-bbba-3994915a40e3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.735856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a30a4ce-8e4f-4255-a3fc-a7958012350e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.737198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a4ce-8e4f-4255-a3fc-a7958012350e service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.737517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a4ce-8e4f-4255-a3fc-a7958012350e service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.737950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a30a4ce-8e4f-4255-a3fc-a7958012350e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.748125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a4ce-8e4f-4255-a3fc-a7958012350e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.749620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a30a4ce-8e4f-4255-a3fc-a7958012350e service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.760425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1391/2736] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:57 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:12:57.807898 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:12:57.818687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9a0d81-2336-471e-9131-8f5ea35f9490 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:12:57.822700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:12:57.823366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:12:57.823673 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:12:57.823944 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:12:57.882535 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:12:57.883999 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:12:57.894190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1346/2737] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:12:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:00.070757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:00.071116 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:00.071227 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:00.090535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] MFA Rules not processed for user `7b6534d42be64c3cb311d384458bfae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:00.091667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:00.136759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:00.152070 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1392/2738] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:01.301922 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:01.303450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:01.346548 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:01.377779 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d02680-749e-4197-a0b2-07fca2f7480b service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:01.382173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:01.382838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:01.383132 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:01.383408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:01.385892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:01.474567 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:01.476155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fffd1d6b-9cb9-45c6-97ca-1d2c10fda153 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:01.489505 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1347/2739] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:03.746523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:03.816908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d65e3d7-85f1-49a9-806a-2b163ca86d7c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:03.823277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:03.824322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:03.824712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:03.825058 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] PATH_INFO: `/v3/users/99f138f0386c4c3d9c7a02e23da0fcaa` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:03.826361 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99f138f0386c4c3d9c7a02e23da0fcaa)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:03.828352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:04.069732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99f138f0386c4c3d9c7a02e23da0fcaa'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:04.070621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99f138f0386c4c3d9c7a02e23da0fcaa'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:04.071402 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99f138f0386c4c3d9c7a02e23da0fcaa'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:04.086582 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99f138f0386c4c3d9c7a02e23da0fcaa'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:04.094326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99f138f0386c4c3d9c7a02e23da0fcaa'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:04.174679 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1393/2740] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:13:03 2022] DELETE /identity/v3/users/99f138f0386c4c3d9c7a02e23da0fcaa => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:04.179742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:04.180265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:04.180265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:04.206417 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:04.207614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:04.274425 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1348/2741] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:04.287557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:04.289085 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:04.362234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4ff1c2-ddf7-4c55-aac3-19f0782d81b1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:04.366068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:04.366730 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:04.367018 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:04.367296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:04.377313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:04.378859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:04.390015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1394/2742] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:04.984014 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:04.990887 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:05.020651 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a7f46b-5fce-4f54-97e6-a7c28e47741e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:05.029554 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:05.030458 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:05.030742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:05.031015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] PATH_INFO: `/v3/projects/553b3bbdb8334ccca35f8c615c237994` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:05.032871 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] RBAC: Authorizing `identity:delete_project(project_id=553b3bbdb8334ccca35f8c615c237994)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:05.034608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:05.190541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '553b3bbdb8334ccca35f8c615c237994', 'request_id': 'req-110b7668-1b82-41c3-948d-dde33e64e98c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.197830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '553b3bbdb8334ccca35f8c615c237994', 'request_id': 'req-110b7668-1b82-41c3-948d-dde33e64e98c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.206163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '553b3bbdb8334ccca35f8c615c237994', 'request_id': 'req-110b7668-1b82-41c3-948d-dde33e64e98c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.212754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:05.328709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] The token cache is being invalidate because project 553b3bbdb8334ccca35f8c615c237994 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:13:05.329094 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.329921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.332930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1349/2743] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:13:04 2022] DELETE /identity/v3/projects/553b3bbdb8334ccca35f8c615c237994 => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:05.336973 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:05.358205 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6fa9ac7-76d7-46a1-a78b-bebfe356731e service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:05.365897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:05.441128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:05.442234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:05.442524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:05.442963 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] PATH_INFO: `/v3/users/ffa988145873427c983d5106e876d6dd` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:05.444531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=ffa988145873427c983d5106e876d6dd)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:05.446496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:05.670391 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ffa988145873427c983d5106e876d6dd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.671677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ffa988145873427c983d5106e876d6dd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.672289 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ffa988145873427c983d5106e876d6dd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.687178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ffa988145873427c983d5106e876d6dd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.698380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a00fd930-bfd6-4626-8d04-8819d9acb6fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ffa988145873427c983d5106e876d6dd'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:05.803408 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1395/2744] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:13:05 2022] DELETE /identity/v3/users/ffa988145873427c983d5106e876d6dd => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:06.389376 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:06.394972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:06.412494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110b7668-1b82-41c3-948d-dde33e64e98c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:06.506228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:06.507778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:06.508233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:06.508708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] PATH_INFO: `/v3/projects/b73a20391dd245cf8bda708eb7501160` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:06.510780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b73a20391dd245cf8bda708eb7501160)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:06.512925 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:06.626077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b73a20391dd245cf8bda708eb7501160', 'request_id': 'req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:06.633746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b73a20391dd245cf8bda708eb7501160', 'request_id': 'req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:06.640018 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b73a20391dd245cf8bda708eb7501160', 'request_id': 'req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:06.727933 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] The token cache is being invalidate because project b73a20391dd245cf8bda708eb7501160 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:13:06.728220 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:06.728873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c1e6e02c-c021-4351-9bbc-ca7046dcbaa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:06.730460 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1350/2745] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:13:06 2022] DELETE /identity/v3/projects/b73a20391dd245cf8bda708eb7501160 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:06.735433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:06.735940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:06.736032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:06.755065 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:06.756300 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:06.819148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1396/2746] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:06.881129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:06.881321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:06.881467 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:06.901418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:06.902710 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:06.911951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1351/2747] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:06.916398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:06.927847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce965c62-785b-4b01-8da9-c09ab63c14b8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:06.932762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:06.933488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:06.933778 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:06.934059 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:06.934817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:13:06.935136 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:06.936819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:06.951144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1397/2748] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:13:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:13:06.957081 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:06.966990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf516490-983c-48e8-ae58-21d75a6d05fb None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:06.972193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:06.972910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:06.977851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:06.977851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:06.977851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:06.977851 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.136691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1352/2749] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:13:06 2022] POST /identity/v3/projects => generated 381 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:07.140535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:07.151740 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7761ab-2743-4e96-9f96-fc6bb7d9d404 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:07.164326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:07.217262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:07.218032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.218320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.218601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.219295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:07.221861 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.362869 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1398/2750] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:13:07 2022] POST /identity/v3/users => generated 291 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:07.367234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:07.379415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f04d5-265e-4ff3-97eb-28bcd8302220 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:07.384179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:07.384913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.385204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.385488 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.386284 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:07.387906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.400235 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1353/2751] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:13:07.404413 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:07.414646 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77275098-ae54-4cf2-8c78-ecc6a188444f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:07.419265 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20c54cb-17a2-428b-af3e-cbc8b91e7c21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:07.419929 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c54cb-17a2-428b-af3e-cbc8b91e7c21 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.420215 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c54cb-17a2-428b-af3e-cbc8b91e7c21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.420493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20c54cb-17a2-428b-af3e-cbc8b91e7c21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.421273 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c54cb-17a2-428b-af3e-cbc8b91e7c21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:07.422866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20c54cb-17a2-428b-af3e-cbc8b91e7c21 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.434179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1399/2752] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:13:07.438243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:07.449286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afe39e6-424e-44ab-bd79-2592f3406e4b admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:07.454795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:07.456372 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.456697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.456982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] PATH_INFO: `/v3/projects/6e66e2609bce4909a07bc5dee764b51e/users/c89a449e210b4350a688ef78972db8d6/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.472379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e66e2609bce4909a07bc5dee764b51e, user_id=c89a449e210b4350a688ef78972db8d6, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:07.475088 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.525493 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1354/2753] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:13:07 2022] PUT /identity/v3/projects/6e66e2609bce4909a07bc5dee764b51e/users/c89a449e210b4350a688ef78972db8d6/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:07.535956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.536259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.539848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.570514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] MFA Rules not processed for user `c89a449e210b4350a688ef78972db8d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:07.571795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:07.644764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1400/2754] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:07.649421 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:07.676147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:07.713568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07152587-2571-475a-9a91-99c10f04572a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:07.719019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:07.719694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.720008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.720289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.722286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:07.724344 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.810512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1355/2755] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:13:07 2022] POST /identity/v3/projects => generated 387 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:07.814622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:07.826142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea85acf3-afd7-4829-8c87-754b59b8cf01 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:07.885557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:07.886183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:07.886514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:07.886791 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:07.887496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:07.889494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:07.992610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1401/2756] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:13:07 2022] POST /identity/v3/users => generated 300 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:07.996894 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:08.009097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c14c7bc1-f000-4554-994e-a3fc9bb36692 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:08.014076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:08.014759 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.015048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.015329 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.016065 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:08.017734 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:08.029968 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1356/2757] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:07 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:13:08.034351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:08.044835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d593481b-516b-403b-83f5-71c1cd9c534f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:08.050199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:08.051101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.051388 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.051668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] PATH_INFO: `/v3/projects/3a2207ceee3c4c47bcca049869e64f5e/users/d919cad29f7b43cb978a97e6bfdd1629/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.063345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:08.068090 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a2207ceee3c4c47bcca049869e64f5e, user_id=d919cad29f7b43cb978a97e6bfdd1629, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:08.071139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:08.134150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1402/2758] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:13:08 2022] PUT /identity/v3/projects/3a2207ceee3c4c47bcca049869e64f5e/users/d919cad29f7b43cb978a97e6bfdd1629/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:13:08.138300 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:08.149558 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3867edf8-543c-463e-a8a5-325289544568 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:08.209555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2400beb-eda2-42a1-8559-26c9dd0f97b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:08.210231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2400beb-eda2-42a1-8559-26c9dd0f97b2 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.210518 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2400beb-eda2-42a1-8559-26c9dd0f97b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.210799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2400beb-eda2-42a1-8559-26c9dd0f97b2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.211534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2400beb-eda2-42a1-8559-26c9dd0f97b2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:08.213184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2400beb-eda2-42a1-8559-26c9dd0f97b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:08.224099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1357/2759] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:08 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:13:08.228255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:08.239463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c41522-c8d1-427c-b9e2-b5a456acc06d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:08.244207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:08.245105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.245394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.245674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] PATH_INFO: `/v3/projects/3a2207ceee3c4c47bcca049869e64f5e/users/d919cad29f7b43cb978a97e6bfdd1629/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.247644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3a2207ceee3c4c47bcca049869e64f5e, user_id=d919cad29f7b43cb978a97e6bfdd1629, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:08.250326 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:08.261818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1403/2760] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:13:08 2022] PUT /identity/v3/projects/3a2207ceee3c4c47bcca049869e64f5e/users/d919cad29f7b43cb978a97e6bfdd1629/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:08.274069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.274365 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.274654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.304501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] MFA Rules not processed for user `d919cad29f7b43cb978a97e6bfdd1629`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:08.305775 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:08.392730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1358/2761] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:08.404216 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:08.406514 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:08.465825 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cad99-48c7-4219-9f8c-6259f8ce8343 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:08.469496 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d71c5-e2de-4bdf-914a-f344d43a0b91 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:08.470154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d71c5-e2de-4bdf-914a-f344d43a0b91 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.470444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d71c5-e2de-4bdf-914a-f344d43a0b91 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.470728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d71c5-e2de-4bdf-914a-f344d43a0b91 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.480555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d71c5-e2de-4bdf-914a-f344d43a0b91 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:08.482037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d71c5-e2de-4bdf-914a-f344d43a0b91 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:08.492491 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1404/2762] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:08 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:08.636610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:08.672360 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:08.744468 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541bc268-7a72-44b2-b724-48a8af489e79 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:08.753050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:08.754292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:08.754786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:08.755397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] PATH_INFO: `/v3/users/c89a449e210b4350a688ef78972db8d6` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:08.757512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c89a449e210b4350a688ef78972db8d6)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:08.760924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:09.039522 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c89a449e210b4350a688ef78972db8d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.048623 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c89a449e210b4350a688ef78972db8d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.062624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c89a449e210b4350a688ef78972db8d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.063487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c89a449e210b4350a688ef78972db8d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.145451 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c89a449e210b4350a688ef78972db8d6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.152066 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1359/2763] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:13:08 2022] DELETE /identity/v3/users/c89a449e210b4350a688ef78972db8d6 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:09.156674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:09.157178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:09.157271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:09.176269 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:09.177527 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:09.238250 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1405/2764] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:09.247854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:09.249583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:09.308495 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289d0bb6-18ea-4a5c-9176-ae628ec373e4 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:09.312112 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:09.312797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:09.313087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:09.313392 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:09.323979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:09.325466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:09.335524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1360/2765] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:09.822373 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:09.823910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:09.843370 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3550a0-bfc9-4a9f-8a9f-56451532fcc2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:09.848432 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:09.849392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:09.849672 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:09.849948 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] PATH_INFO: `/v3/projects/6e66e2609bce4909a07bc5dee764b51e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:09.851160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e66e2609bce4909a07bc5dee764b51e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:09.853132 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:09.982028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e66e2609bce4909a07bc5dee764b51e', 'request_id': 'req-407a509a-9d5e-4799-83f9-026600e8be6b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.988506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e66e2609bce4909a07bc5dee764b51e', 'request_id': 'req-407a509a-9d5e-4799-83f9-026600e8be6b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:09.994589 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e66e2609bce4909a07bc5dee764b51e', 'request_id': 'req-407a509a-9d5e-4799-83f9-026600e8be6b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.099416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] The token cache is being invalidate because project 6e66e2609bce4909a07bc5dee764b51e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:13:10.099416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.099416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.099416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1406/2766] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:13:09 2022] DELETE /identity/v3/projects/6e66e2609bce4909a07bc5dee764b51e => generated 0 bytes in 276 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:10.103237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:10.115356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c22b60-1452-400e-b8dd-b4eba9a9bb95 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:10.159316 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:10.177643 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:10.178533 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:10.178825 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:10.179071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] PATH_INFO: `/v3/users/d919cad29f7b43cb978a97e6bfdd1629` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:10.180269 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d919cad29f7b43cb978a97e6bfdd1629)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:10.182312 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:10.478684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd919cad29f7b43cb978a97e6bfdd1629'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.485809 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd919cad29f7b43cb978a97e6bfdd1629'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.497767 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd919cad29f7b43cb978a97e6bfdd1629'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.498504 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd919cad29f7b43cb978a97e6bfdd1629'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.590878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-052ad408-856b-4e87-801a-d8e7526ecc65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd919cad29f7b43cb978a97e6bfdd1629'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:10.598616 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1361/2767] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:13:10 2022] DELETE /identity/v3/users/d919cad29f7b43cb978a97e6bfdd1629 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:11.061199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.062694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:11.076713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:11.077298 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407a509a-9d5e-4799-83f9-026600e8be6b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.138061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.138858 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.139143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.139422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] PATH_INFO: `/v3/projects/3a2207ceee3c4c47bcca049869e64f5e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.140666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=3a2207ceee3c4c47bcca049869e64f5e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.142602 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:11.279427 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3a2207ceee3c4c47bcca049869e64f5e', 'request_id': 'req-c8f497ea-7656-49dc-8260-5339db6782cf'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:11.286277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3a2207ceee3c4c47bcca049869e64f5e', 'request_id': 'req-c8f497ea-7656-49dc-8260-5339db6782cf'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:11.291815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3a2207ceee3c4c47bcca049869e64f5e', 'request_id': 'req-c8f497ea-7656-49dc-8260-5339db6782cf'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:11.368374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] The token cache is being invalidate because project 3a2207ceee3c4c47bcca049869e64f5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:13:11.368725 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:11.369346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c8f497ea-7656-49dc-8260-5339db6782cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:11.370937 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1407/2768] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:13:11 2022] DELETE /identity/v3/projects/3a2207ceee3c4c47bcca049869e64f5e => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:13:11.375909 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.376293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.376501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.401576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:11.401859 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:11.464753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1362/2769] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:11.472642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.472936 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.473219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.492327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:11.493587 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:11.501359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1408/2770] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:11.506250 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.515997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d56ecd0-623f-441b-8132-ec2354555c7a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.521757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.533401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.533401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.533401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.533401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:13:11.533401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.533401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:11.538801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1363/2771] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 23 14:13:11.543352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.553104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f36901c-1b6d-46f1-82f8-694b4d0c57cc None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.557762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.558426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.558713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.558992 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.559680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.561659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:11.650699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1409/2772] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:13:11 2022] POST /identity/v3/projects => generated 381 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:11.655056 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.666293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf466ef-48f8-4e42-849c-f5d071468b7a admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.716397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:11.730069 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.730764 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.731055 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.731337 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.732793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.734744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:11.814556 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1364/2773] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:13:11 2022] POST /identity/v3/users => generated 291 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:11.819569 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.830113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e23ede-5643-47ca-b842-85ecdebfadef admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.835788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.836464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.836786 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.837071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.837808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.839416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:11.851986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1410/2774] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:13:11.856385 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.867521 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b578b6-bd2d-4764-919e-53aea7925eaa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.872237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5a7c16-c680-4ecf-8eb9-62da919673fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.872930 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7c16-c680-4ecf-8eb9-62da919673fb admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.873217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7c16-c680-4ecf-8eb9-62da919673fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.873498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5a7c16-c680-4ecf-8eb9-62da919673fb admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.874268 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a7c16-c680-4ecf-8eb9-62da919673fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.875938 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5a7c16-c680-4ecf-8eb9-62da919673fb admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:11.888302 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1365/2775] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:13:11.892635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:11.903612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fa1e04-0d4d-4a58-9a48-757db64daeab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:11.912333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:11.912333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:11.912333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:11.912333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] PATH_INFO: `/v3/projects/defa34a797b34a9695df39e950a1cb31/users/fa56450609e34c2e9ba410608e3e7482/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:11.930879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] RBAC: Authorizing `identity:create_grant(project_id=defa34a797b34a9695df39e950a1cb31, user_id=fa56450609e34c2e9ba410608e3e7482, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:11.933694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.015029 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1411/2776] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:13:11 2022] PUT /identity/v3/projects/defa34a797b34a9695df39e950a1cb31/users/fa56450609e34c2e9ba410608e3e7482/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:12.026090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.026383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.026692 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.055052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] MFA Rules not processed for user `fa56450609e34c2e9ba410608e3e7482`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:12.056272 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:12.123895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1366/2777] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:13:12.128426 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.159252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:12.191609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698992-8a55-4dab-bfa9-7bb4c0da312b admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:12.196379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:12.197208 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.197499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.197776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.198481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:12.200414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.285920 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1412/2778] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:13:12 2022] POST /identity/v3/projects => generated 383 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:13:12.290219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.301463 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e142f7d4-32dd-4ce9-a4a4-bee5d4ee0c29 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:12.360199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:12.360912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.361204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.361882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.362253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:12.364754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.489994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1367/2779] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:13:12 2022] POST /identity/v3/users => generated 298 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:13:12.492731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.504650 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad1fc95-2edf-4012-a460-8e6b93ab24c3 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:12.509675 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:12.510540 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.510835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.511114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.511862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:12.513516 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.526653 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1413/2780] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:13:12.531031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.542380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4605bf63-f727-4652-88b2-d99477e17331 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:12.547262 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:12.548155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.548436 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.548761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] PATH_INFO: `/v3/projects/d90fde406df146879c6119c8ee497666/users/13c4a1d23a4a46389defdfd9ba9d751f/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.573304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90fde406df146879c6119c8ee497666, user_id=13c4a1d23a4a46389defdfd9ba9d751f, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:12.577942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.666015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1368/2781] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:13:12 2022] PUT /identity/v3/projects/d90fde406df146879c6119c8ee497666/users/13c4a1d23a4a46389defdfd9ba9d751f/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:13:12.674976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.685505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b4e0a2-be56-4b2f-8dc2-feb7797f4070 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:12.747330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc63a48-13da-4583-a194-78f52d57bbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:12.748014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc63a48-13da-4583-a194-78f52d57bbad admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.748219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc63a48-13da-4583-a194-78f52d57bbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.748502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc63a48-13da-4583-a194-78f52d57bbad admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.749331 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc63a48-13da-4583-a194-78f52d57bbad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:12.750969 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc63a48-13da-4583-a194-78f52d57bbad admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.762106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1414/2782] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:13:12 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:13:12.766380 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.778008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f742bc70-4653-439b-a57f-1e8942834513 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:12.783353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:12.784256 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.784536 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.784850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] PATH_INFO: `/v3/projects/d90fde406df146879c6119c8ee497666/users/13c4a1d23a4a46389defdfd9ba9d751f/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.787179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] RBAC: Authorizing `identity:create_grant(project_id=d90fde406df146879c6119c8ee497666, user_id=13c4a1d23a4a46389defdfd9ba9d751f, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:12.789994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:12.801822 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1369/2783] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:13:12 2022] PUT /identity/v3/projects/d90fde406df146879c6119c8ee497666/users/13c4a1d23a4a46389defdfd9ba9d751f/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:12.813362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:12.813661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:12.813943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:12.843341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] MFA Rules not processed for user `13c4a1d23a4a46389defdfd9ba9d751f`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:12.844598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:12.924195 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1415/2784] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:12 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:12.936662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:12.938672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:12.956031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:13.007038 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4386b00-86f8-4237-9d00-af82f06c8bef admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:13.010654 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:13.011315 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:13.011601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:13.011879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:13.022137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:13.023658 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:13.035258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1370/2785] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:12 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:13.179804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:13.190050 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f49f20-a15c-4df2-9afe-b2728c48bb12 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:13.195104 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf8638-99d7-48b8-912b-8812dcfaabee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:13.195761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8638-99d7-48b8-912b-8812dcfaabee service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:13.196047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8638-99d7-48b8-912b-8812dcfaabee service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:13.196327 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf8638-99d7-48b8-912b-8812dcfaabee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:13.218279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:13.257820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8638-99d7-48b8-912b-8812dcfaabee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:13.259291 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf8638-99d7-48b8-912b-8812dcfaabee service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:13.268458 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1416/2786] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:13 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:13.799292 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:13.862573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0023ca-89eb-4a95-8f1d-56882cdac2c0 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:13.867739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:13.868642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:13.868931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:13.869202 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] PATH_INFO: `/v3/users/fa56450609e34c2e9ba410608e3e7482` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:13.870377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa56450609e34c2e9ba410608e3e7482)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:13.872273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:14.078725 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa56450609e34c2e9ba410608e3e7482'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:14.085985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa56450609e34c2e9ba410608e3e7482'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:14.098706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa56450609e34c2e9ba410608e3e7482'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:14.099339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa56450609e34c2e9ba410608e3e7482'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:14.174071 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa56450609e34c2e9ba410608e3e7482'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:14.181394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1371/2787] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:13:13 2022] DELETE /identity/v3/users/fa56450609e34c2e9ba410608e3e7482 => generated 0 bytes in 383 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:14.186579 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:14.187079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:14.187166 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:14.205142 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:13:14.206636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:14.280001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1417/2788] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:13:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:13:14.290644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:14.292136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:14.325251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:14.353229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c15b4e-8ead-40e1-a351-5022abbb6076 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:14.357414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:14.358128 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:14.358407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:14.358818 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:14.370464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:14.371975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:14.382647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1372/2789] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:13:14.855411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:14.857022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:14.875286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e986c3e4-6a17-4603-bdae-36f04dab7907 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:14.881838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:14.892068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:14.892068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:14.892068 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] PATH_INFO: `/v3/projects/defa34a797b34a9695df39e950a1cb31` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:14.896972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=defa34a797b34a9695df39e950a1cb31)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:14.899101 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:15.039285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'defa34a797b34a9695df39e950a1cb31', 'request_id': 'req-9eacd4d5-b56b-4835-89a6-0a28b175faf1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.045950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'defa34a797b34a9695df39e950a1cb31', 'request_id': 'req-9eacd4d5-b56b-4835-89a6-0a28b175faf1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.052239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'defa34a797b34a9695df39e950a1cb31', 'request_id': 'req-9eacd4d5-b56b-4835-89a6-0a28b175faf1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.122203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] The token cache is being invalidate because project defa34a797b34a9695df39e950a1cb31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:13:15.122422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.123063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.124642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1418/2790] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:13:14 2022] DELETE /identity/v3/projects/defa34a797b34a9695df39e950a1cb31 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:15.129102 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:15.139327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625afcd5-207d-4ea6-af92-06246dc038f5 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:15.199377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:15.200298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:15.200610 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:15.200921 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] PATH_INFO: `/v3/users/13c4a1d23a4a46389defdfd9ba9d751f` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:15.202146 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13c4a1d23a4a46389defdfd9ba9d751f)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:15.204084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:15.382766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13c4a1d23a4a46389defdfd9ba9d751f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.389624 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13c4a1d23a4a46389defdfd9ba9d751f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.402107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13c4a1d23a4a46389defdfd9ba9d751f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.402808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13c4a1d23a4a46389defdfd9ba9d751f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.482154 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13c4a1d23a4a46389defdfd9ba9d751f'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:15.489724 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1373/2791] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:13:15 2022] DELETE /identity/v3/users/13c4a1d23a4a46389defdfd9ba9d751f => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:15.906821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:15.908266 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:15.928118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eacd4d5-b56b-4835-89a6-0a28b175faf1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:15.990959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:16.008156 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:16.009407 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:16.009870 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:16.010313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] PATH_INFO: `/v3/projects/d90fde406df146879c6119c8ee497666` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:16.012187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=d90fde406df146879c6119c8ee497666)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:16.015152 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:16.150223 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd90fde406df146879c6119c8ee497666', 'request_id': 'req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:16.156557 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd90fde406df146879c6119c8ee497666', 'request_id': 'req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:16.162063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd90fde406df146879c6119c8ee497666', 'request_id': 'req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:16.225178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] The token cache is being invalidate because project d90fde406df146879c6119c8ee497666 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:13:16.225482 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:16.226153 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:13:16.227755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1419/2792] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:13:15 2022] DELETE /identity/v3/projects/d90fde406df146879c6119c8ee497666 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:13:21.017729 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:13:21.019288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:21.084197 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4869b8aa-fcda-45b4-85e9-73699556bc70 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:13:21.088072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:21.139631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:13:21.140296 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:13:21.140608 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:13:21.140901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:13:21.142746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:13:21.212900 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:13:21.214401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:13:21.226394 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1374/2793] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:13:21 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:13:26.221390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=30,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:13:31.221201 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=47 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:14:06.688615 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:06.690177 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:06.704180 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cafe97-2e73-4eaa-ad9a-ea41d7c6f3ab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:06.709632 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:06.710343 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:06.710630 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:06.710914 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:06.712847 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:06.724924 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:06.726437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:06.738489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1420/2794] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:14:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:14:16.732746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:14:39.606388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:39.608758 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:39.700184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3049855-904c-41b2-a68f-30f7d7258d16 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:39.706158 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:39.706834 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:39.707118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:39.707399 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:39.709538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:39.725273 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:39.728352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:39.741789 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1375/2795] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:14:39 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:14:45.344003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:45.345455 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:45.361107 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5cd03d-07ac-4fb7-aba0-21ead0d46ce7 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:45.364900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:45.365550 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:45.365832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:45.366112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:45.367882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:45.440148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:45.440148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:45.449061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1421/2796] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:14:49.244679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:49.248062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:49.321061 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d5879b-fa40-4d9f-a064-041bb0f191da service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:49.326229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:49.326917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:49.327203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:49.327478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:49.329305 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:49.342283 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:49.346317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:49.354865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1376/2797] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:14:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:14:49.375958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:49.377591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:49.390112 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeba0410-ce71-4f30-9b17-7d61f93dba75 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:49.394707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:49.396139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:49.396420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:49.396734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] PATH_INFO: `/v3/users/af29f227080147a1a157f88842d39483` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:49.397941 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af29f227080147a1a157f88842d39483)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:49.399865 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:49.559399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af29f227080147a1a157f88842d39483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:49.560057 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af29f227080147a1a157f88842d39483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:49.560723 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af29f227080147a1a157f88842d39483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:49.573701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af29f227080147a1a157f88842d39483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:49.580744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-78e85a8d-2405-4884-a61e-e109a5995d74 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af29f227080147a1a157f88842d39483'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:49.654198 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1422/2798] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:14:49 2022] DELETE /identity/v3/users/af29f227080147a1a157f88842d39483 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:14:49.913671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:49.925414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9840e50d-67ba-4432-ae49-cd1134b3525e service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:49.985274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:49.986138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:49.986418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:49.986690 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] PATH_INFO: `/v3/projects/c4fb50a6ddee45278948b5fed215597b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:49.987858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4fb50a6ddee45278948b5fed215597b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:49.990090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.009188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:14:50.150691 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4fb50a6ddee45278948b5fed215597b', 'request_id': 'req-1cad0273-2310-4451-9fa9-8063e82d96a2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:50.159270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4fb50a6ddee45278948b5fed215597b', 'request_id': 'req-1cad0273-2310-4451-9fa9-8063e82d96a2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:50.167449 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4fb50a6ddee45278948b5fed215597b', 'request_id': 'req-1cad0273-2310-4451-9fa9-8063e82d96a2'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:50.248660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] The token cache is being invalidate because project c4fb50a6ddee45278948b5fed215597b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:14:50.249331 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:50.249655 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-1cad0273-2310-4451-9fa9-8063e82d96a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:14:50.251407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1377/2799] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:14:49 2022] DELETE /identity/v3/projects/c4fb50a6ddee45278948b5fed215597b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:14:50.258066 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.258359 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.258641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.277442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:14:50.278665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:50.338399 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1423/2800] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:14:50.346481 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939515ce-a373-4aab-aee1-9113a328274a None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.346776 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939515ce-a373-4aab-aee1-9113a328274a None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.347062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939515ce-a373-4aab-aee1-9113a328274a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.366588 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-939515ce-a373-4aab-aee1-9113a328274a None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:14:50.367813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-939515ce-a373-4aab-aee1-9113a328274a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:50.375970 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1378/2801] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:14:50.380939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:50.390481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c875b5-9518-4bba-8dc6-e15cf4ee1967 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:50.394996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:50.395651 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.395939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.396213 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.396968 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:14:50.397276 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:50.398854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.400347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:14:50.414189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1424/2802] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:14:50.416633 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939515ce-a373-4aab-aee1-9113a328274a None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:50.428101 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939515ce-a373-4aab-aee1-9113a328274a None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:50.433672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:50.434374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.434662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.434939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.435709 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:50.437689 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.525118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1379/2803] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:14:50 2022] POST /identity/v3/projects => generated 375 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:14:50.529498 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:50.540715 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4b8253-79c8-428c-9ee2-9fa80ef44ad4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:50.597155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:50.597810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.598664 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.599163 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.599704 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:50.601635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.684304 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1425/2804] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:14:50 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:14:50.688357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:50.698934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78bb16ec-222e-4529-8c81-031c998e50c6 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:50.703632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:50.704346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.704678 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.708036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.708036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:50.708036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.718738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1380/2805] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:14:50.723735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:50.733824 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7de064-65d9-48ed-bd30-fc0f42e874f5 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:50.738411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e5bd905-f33d-4603-af3a-83f863898cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:50.739975 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd905-f33d-4603-af3a-83f863898cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.740258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd905-f33d-4603-af3a-83f863898cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.740531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e5bd905-f33d-4603-af3a-83f863898cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.741286 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bd905-f33d-4603-af3a-83f863898cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:50.742862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e5bd905-f33d-4603-af3a-83f863898cc2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.752862 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1426/2806] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:14:50.757103 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:50.767583 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17968a-fc33-41df-a5e0-d7463c618ad5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:50.773738 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:50.774631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.774911 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.775188 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] PATH_INFO: `/v3/projects/c28d5c5216674cab891397f4ecb07892/users/7bd9b179f3ac4044824d90bfd15febae/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.790829 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c28d5c5216674cab891397f4ecb07892, user_id=7bd9b179f3ac4044824d90bfd15febae, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:50.793544 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:50.856171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1381/2807] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:14:50 2022] PUT /identity/v3/projects/c28d5c5216674cab891397f4ecb07892/users/7bd9b179f3ac4044824d90bfd15febae/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:14:50.867261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:50.867590 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:50.867877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:50.895620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] MFA Rules not processed for user `7bd9b179f3ac4044824d90bfd15febae`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:14:50.896803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:50.962721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1427/2808] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:14:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:14:50.967161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.032113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2b8cfd-c034-41f3-905d-08c400726982 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.037716 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.038298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.038589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.038872 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.039582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.041540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.137282 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1382/2809] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:14:50 2022] POST /identity/v3/projects => generated 375 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:14:51.141351 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.156075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae2e90c-613f-461e-85c2-0986f4786748 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.221731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.222487 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.222784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.223061 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.223762 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.226572 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.234322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:14:51.353566 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1428/2810] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:14:51 2022] POST /identity/v3/users => generated 294 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:14:51.357631 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.369288 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce3dc3e-5708-4ce8-9822-8088f2827d60 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.374153 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.378323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.378323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.378323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.378323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.378323 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.389327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1383/2811] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:14:51.393716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.404515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceff4375-72df-4340-a2b5-21c8ec32df80 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.410866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.410866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.410866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.410866 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] PATH_INFO: `/v3/projects/33ca49c3182d4e9fab5838392f6fe36e/users/59e089abd9f946a4a1070015c798df60/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.426515 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ca49c3182d4e9fab5838392f6fe36e, user_id=59e089abd9f946a4a1070015c798df60, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.429449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.546846 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1429/2812] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:14:51 2022] PUT /identity/v3/projects/33ca49c3182d4e9fab5838392f6fe36e/users/59e089abd9f946a4a1070015c798df60/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:14:51.550804 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.560942 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eea0a3-064b-48ed-8b3e-aecf91a85bfa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.572515 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:14:51.619072 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83fe75-9339-4c01-9cf2-9f397d670222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.619744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83fe75-9339-4c01-9cf2-9f397d670222 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.620032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83fe75-9339-4c01-9cf2-9f397d670222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.620313 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83fe75-9339-4c01-9cf2-9f397d670222 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.621091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83fe75-9339-4c01-9cf2-9f397d670222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.622699 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83fe75-9339-4c01-9cf2-9f397d670222 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.633095 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1384/2813] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:14:51.637907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.648382 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745ed012-d331-48aa-8960-f05e4c3e3f91 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.653642 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.654522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.654813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.655105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] PATH_INFO: `/v3/projects/33ca49c3182d4e9fab5838392f6fe36e/users/59e089abd9f946a4a1070015c798df60/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.657144 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] RBAC: Authorizing `identity:create_grant(project_id=33ca49c3182d4e9fab5838392f6fe36e, user_id=59e089abd9f946a4a1070015c798df60, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.659981 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.671098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1430/2814] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:14:51 2022] PUT /identity/v3/projects/33ca49c3182d4e9fab5838392f6fe36e/users/59e089abd9f946a4a1070015c798df60/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:14:51.694194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.694571 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.694858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.722198 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] MFA Rules not processed for user `59e089abd9f946a4a1070015c798df60`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:14:51.723437 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:51.792771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1385/2815] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:14:51 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:14:51.803480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:51.805501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:14:51.866430 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce1a058-cd8f-45b6-a2ae-771581bb029e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:51.870398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:51.871046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:51.871320 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:51.871592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:51.928415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:51.929884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:51.939157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1431/2816] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:14:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:14:59.232175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:14:59.243075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1debd60-2f7d-4aa4-9d30-2e2de95952d9 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:14:59.247216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719dc1c9-35a6-4579-826f-b88ebdc1d89d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:14:59.247885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc1c9-35a6-4579-826f-b88ebdc1d89d service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:14:59.248173 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc1c9-35a6-4579-826f-b88ebdc1d89d service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:14:59.248452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719dc1c9-35a6-4579-826f-b88ebdc1d89d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:14:59.258838 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dc1c9-35a6-4579-826f-b88ebdc1d89d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:14:59.260353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-719dc1c9-35a6-4579-826f-b88ebdc1d89d service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:14:59.271497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1386/2817] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:14:59 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:01.935071 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:INSERT=4,keystone:SELECT=65 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:09.266381 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=62 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:30.452776 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:30.516105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4613b886-721e-45f4-9ed4-d2d4dab21378 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:30.520728 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:30.521701 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:30.521980 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:30.522272 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] PATH_INFO: `/v3/users/7bd9b179f3ac4044824d90bfd15febae` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:30.523425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bd9b179f3ac4044824d90bfd15febae)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:30.525367 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:30.793988 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bd9b179f3ac4044824d90bfd15febae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:30.794547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bd9b179f3ac4044824d90bfd15febae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:30.795148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bd9b179f3ac4044824d90bfd15febae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:30.807989 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bd9b179f3ac4044824d90bfd15febae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:30.816203 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bd9b179f3ac4044824d90bfd15febae'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:30.895804 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1432/2818] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:30 2022] DELETE /identity/v3/users/7bd9b179f3ac4044824d90bfd15febae => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:30.900589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:30.902917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:30.902917 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:30.919803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:30.921317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:30.995512 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1387/2819] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:31.006222 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:31.007700 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:31.065832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-852d1f2a-3150-4186-bc69-571169402062 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:31.069434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:31.070082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:31.070365 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:31.070644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:31.080190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:31.081939 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:31.091417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1433/2820] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:31.564878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:31.566455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:31.581251 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7392ef-e553-473e-8665-5fedc09e8024 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:31.586043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:31.586920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:31.587808 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:31.588098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] PATH_INFO: `/v3/projects/c28d5c5216674cab891397f4ecb07892` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:31.589543 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] RBAC: Authorizing `identity:delete_project(project_id=c28d5c5216674cab891397f4ecb07892)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:31.591664 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:31.724501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c28d5c5216674cab891397f4ecb07892', 'request_id': 'req-7fccadb4-60c5-4221-a707-22a00ef3a39c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:31.730952 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c28d5c5216674cab891397f4ecb07892', 'request_id': 'req-7fccadb4-60c5-4221-a707-22a00ef3a39c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:31.737077 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c28d5c5216674cab891397f4ecb07892', 'request_id': 'req-7fccadb4-60c5-4221-a707-22a00ef3a39c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:31.848285 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] The token cache is being invalidate because project c28d5c5216674cab891397f4ecb07892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:31.848484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:31.849111 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:31.850708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1388/2821] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:31 2022] DELETE /identity/v3/projects/c28d5c5216674cab891397f4ecb07892 => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:31.854713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:31.864754 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f120312e-a671-495c-8343-68af41e330e8 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:31.884724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:31.938412 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:31.939386 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:31.939677 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:31.939956 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] PATH_INFO: `/v3/users/59e089abd9f946a4a1070015c798df60` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:31.941262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] RBAC: Authorizing `identity:delete_user(user_id=59e089abd9f946a4a1070015c798df60)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:31.943183 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:32.131971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '59e089abd9f946a4a1070015c798df60'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.132735 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '59e089abd9f946a4a1070015c798df60'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.133360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '59e089abd9f946a4a1070015c798df60'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.145434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '59e089abd9f946a4a1070015c798df60'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.152339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6c505022-a0f9-406f-93d1-c9d0e13f5a6d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '59e089abd9f946a4a1070015c798df60'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.239293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1434/2822] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:31 2022] DELETE /identity/v3/users/59e089abd9f946a4a1070015c798df60 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:32.684440 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:32.686034 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:32.699565 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fccadb4-60c5-4221-a707-22a00ef3a39c admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:32.762742 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:32.763603 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:32.763888 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:32.764181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] PATH_INFO: `/v3/projects/33ca49c3182d4e9fab5838392f6fe36e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:32.765427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33ca49c3182d4e9fab5838392f6fe36e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:32.767327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:32.856317 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:32.873298 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33ca49c3182d4e9fab5838392f6fe36e', 'request_id': 'req-633c787f-6cb5-4618-89ec-44b8e9f62558'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.880113 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33ca49c3182d4e9fab5838392f6fe36e', 'request_id': 'req-633c787f-6cb5-4618-89ec-44b8e9f62558'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.885715 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33ca49c3182d4e9fab5838392f6fe36e', 'request_id': 'req-633c787f-6cb5-4618-89ec-44b8e9f62558'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.974810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] The token cache is being invalidate because project 33ca49c3182d4e9fab5838392f6fe36e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:32.975013 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.975640 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-633c787f-6cb5-4618-89ec-44b8e9f62558 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:32.977248 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1389/2823] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:32 2022] DELETE /identity/v3/projects/33ca49c3182d4e9fab5838392f6fe36e => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:32.982336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:32.982835 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:32.982930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.002009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:33.003299 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:33.067635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1435/2824] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:15:33.075357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.075649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.075931 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.095770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:33.097093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:33.105093 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1390/2825] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:33.109504 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.119212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e581-dd5c-4cc9-9c79-f7cb97508c2c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.123918 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.124818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.125111 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.125398 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.126131 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:15:33.126437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.128154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.139761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1436/2826] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:15:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:15:33.145099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.154388 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180a2be-f5db-48a0-b91c-e6bbde448fcc None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.160303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.160994 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.161498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.161606 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.162243 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.164171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.319793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1391/2827] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:33 2022] POST /identity/v3/projects => generated 407 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:33.323644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.338170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d266e4-cd9f-4d50-bda1-35697e5259ab admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.366492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:33.399010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.399818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.400100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.400379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.401102 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.403063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.472155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1437/2828] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:33 2022] POST /identity/v3/users => generated 304 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:33.476297 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.487411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e5a717-701a-42d5-8f5a-aa88ad17c90e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.492164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.492874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.493246 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.493541 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.494257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.495866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.507085 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1392/2829] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:33.511295 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.522106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f611147e-683f-4274-a9ee-d4bbb6b24c0d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.526697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb85f36-ad15-4d47-8abe-ea82b9d3048f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.527347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85f36-ad15-4d47-8abe-ea82b9d3048f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.527584 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85f36-ad15-4d47-8abe-ea82b9d3048f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.527902 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb85f36-ad15-4d47-8abe-ea82b9d3048f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.528601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb85f36-ad15-4d47-8abe-ea82b9d3048f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.530175 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb85f36-ad15-4d47-8abe-ea82b9d3048f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.540191 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1438/2830] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:33.544461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.554561 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06aa8eaf-a8da-4e42-a3c4-d6e5ebcc9e9d admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.559416 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.560307 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.560627 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.560910 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] PATH_INFO: `/v3/projects/505b89a40ca0461a9069c7452947ce56/users/85b7714fcffb4297a8c2283b322a2a10/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.576525 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] RBAC: Authorizing `identity:create_grant(project_id=505b89a40ca0461a9069c7452947ce56, user_id=85b7714fcffb4297a8c2283b322a2a10, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.579853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.654461 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1393/2831] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:33 2022] PUT /identity/v3/projects/505b89a40ca0461a9069c7452947ce56/users/85b7714fcffb4297a8c2283b322a2a10/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:33.671160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.671499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.671780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.698411 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] MFA Rules not processed for user `85b7714fcffb4297a8c2283b322a2a10`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:33.699712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:33.765189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1439/2832] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:33.769563 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.831621 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e95fb-d158-4ac6-9118-11fd2895baba admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:33.836167 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:33.836841 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:33.837138 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:33.837401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:33.838096 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:33.839990 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:33.953274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1394/2833] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:33 2022] POST /identity/v3/projects => generated 409 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:33.957600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:33.968857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e863659-3844-42ea-929d-577351d0baf8 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:34.027422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:34.028187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.028378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.028689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.029410 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:34.031910 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:34.100982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1440/2834] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:33 2022] POST /identity/v3/users => generated 311 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:34.105015 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:34.116234 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b1103c-ac60-41ae-ad0d-0d99c7f6add7 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:34.121175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:34.123929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.123929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.123929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.123929 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:34.124845 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:34.135865 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1395/2835] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:15:34.139955 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:34.150283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3597c0-3645-400e-882b-0c5f6d76788e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:34.154801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:34.156333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.156333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.156946 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] PATH_INFO: `/v3/projects/8682d34596554078a1eb4f0a561151cc/users/88aa66a5c2fb47538f208ce878737a1c/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.172074 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8682d34596554078a1eb4f0a561151cc, user_id=88aa66a5c2fb47538f208ce878737a1c, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:34.174676 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:34.243739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1441/2836] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:34 2022] PUT /identity/v3/projects/8682d34596554078a1eb4f0a561151cc/users/88aa66a5c2fb47538f208ce878737a1c/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:34.247784 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:34.258039 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac40c16-80a1-41a8-812a-77713dc101fe admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:34.277584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:34.316330 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c25dfd-6d2b-4d0c-a332-9bca9d5d0865 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:34.317029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c25dfd-6d2b-4d0c-a332-9bca9d5d0865 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.317352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c25dfd-6d2b-4d0c-a332-9bca9d5d0865 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.317656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c25dfd-6d2b-4d0c-a332-9bca9d5d0865 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.318363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c25dfd-6d2b-4d0c-a332-9bca9d5d0865 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:34.319985 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c25dfd-6d2b-4d0c-a332-9bca9d5d0865 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:34.330118 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1396/2837] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:34 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:34.334711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:34.340625 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:34.346784 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf02ea00-d8d7-4287-acd9-f25b40a6974d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:34.354795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:34.354795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.354795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.354795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] PATH_INFO: `/v3/projects/8682d34596554078a1eb4f0a561151cc/users/88aa66a5c2fb47538f208ce878737a1c/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.355179 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=8682d34596554078a1eb4f0a561151cc, user_id=88aa66a5c2fb47538f208ce878737a1c, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:34.357600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:34.368748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1442/2838] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:34 2022] PUT /identity/v3/projects/8682d34596554078a1eb4f0a561151cc/users/88aa66a5c2fb47538f208ce878737a1c/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 23 14:15:34.426339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.426626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.426906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.453568 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] MFA Rules not processed for user `88aa66a5c2fb47538f208ce878737a1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:34.454884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:34.523573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1397/2839] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:34 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:34.534125 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:34.535818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:34.594139 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a613a27c-9528-4528-9d07-9219cea586fb admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:34.597922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9091bd1-511b-4010-a94d-544299256686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:34.598574 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9091bd1-511b-4010-a94d-544299256686 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:34.598857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9091bd1-511b-4010-a94d-544299256686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:34.599138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9091bd1-511b-4010-a94d-544299256686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:34.608285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9091bd1-511b-4010-a94d-544299256686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:34.610027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9091bd1-511b-4010-a94d-544299256686 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:34.621236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1443/2840] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:34.993129 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:35.004545 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e0f155-5098-456f-b3e1-35e978c4c2c3 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:35.008979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:35.009484 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:35.009770 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:35.010045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:35.074928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:35.076418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:35.086902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1398/2841] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:44.615605 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:45.082816 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=80 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:45.758919 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:45.759736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:45.760025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:45.778403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] MFA Rules not processed for user `7b6534d42be64c3cb311d384458bfae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:45.779635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:45.835290 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1444/2842] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:46.151492 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:46.153057 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:46.218489 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e654d49e-2148-43f8-aab8-cdecc3c313b6 service cinder] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:46.222660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:46.223384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:46.223670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:46.223948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:46.225810 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:46.235626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:46.237108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68c2998-cc11-41d6-8f67-312553d3c63d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:46.246121 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1399/2843] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:46.274674 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:46.343234 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153d605a-edd3-4785-a8fa-193982b7099c None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:46.349305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:46.350466 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:46.350753 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:46.351032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] PATH_INFO: `/v3/users/85b7714fcffb4297a8c2283b322a2a10` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:46.352283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85b7714fcffb4297a8c2283b322a2a10)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:46.354284 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:46.538046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85b7714fcffb4297a8c2283b322a2a10'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:46.538694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85b7714fcffb4297a8c2283b322a2a10'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:46.539464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85b7714fcffb4297a8c2283b322a2a10'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:46.554713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85b7714fcffb4297a8c2283b322a2a10'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:46.562387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85b7714fcffb4297a8c2283b322a2a10'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:46.660755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1445/2844] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:46 2022] DELETE /identity/v3/users/85b7714fcffb4297a8c2283b322a2a10 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:46.665497 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:46.665819 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:46.666104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:46.692404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:46.693756 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:46.763683 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1400/2845] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:46.773935 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:46.775418 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:46.806881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:46.836207 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c7c45-c20b-42d4-abb1-0bbd9c0b6ca0 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:46.839818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:46.840614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:46.840907 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:46.841187 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:46.850854 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:46.852293 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:46.862155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1446/2846] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:47.100349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:47.101928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:47.211509 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b833efe0-8b75-42f9-b624-58fe56b5517e None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:47.217752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:47.218422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:47.218714 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:47.219309 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:47.221831 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:47.239649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:47.241848 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:47.256978 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1401/2847] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:47 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:47.388758 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:47.390788 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:47.406644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541e65d3-1a85-4e45-a657-28f6cb5a1168 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:47.413820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:47.414805 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:47.415257 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:47.415810 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] PATH_INFO: `/v3/projects/505b89a40ca0461a9069c7452947ce56` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:47.417143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=505b89a40ca0461a9069c7452947ce56)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:47.419148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:47.559106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '505b89a40ca0461a9069c7452947ce56', 'request_id': 'req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:47.565422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '505b89a40ca0461a9069c7452947ce56', 'request_id': 'req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:47.571501 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '505b89a40ca0461a9069c7452947ce56', 'request_id': 'req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:47.667336 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] The token cache is being invalidate because project 505b89a40ca0461a9069c7452947ce56 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:47.667635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:47.668296 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:47.669894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1447/2848] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:47 2022] DELETE /identity/v3/projects/505b89a40ca0461a9069c7452947ce56 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:47.673852 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:47.685237 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166fec1e-d61c-4796-806e-d027c556b966 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:47.719214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:47.746176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:47.747048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:47.747326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:47.747596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] PATH_INFO: `/v3/users/88aa66a5c2fb47538f208ce878737a1c` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:47.748794 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88aa66a5c2fb47538f208ce878737a1c)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:47.750745 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:47.966780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88aa66a5c2fb47538f208ce878737a1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:47.975001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88aa66a5c2fb47538f208ce878737a1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:48.006466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88aa66a5c2fb47538f208ce878737a1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:48.007231 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88aa66a5c2fb47538f208ce878737a1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:48.066573 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-e4399f44-8189-49a9-a1d5-d2ec27e5a516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88aa66a5c2fb47538f208ce878737a1c'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:48.073050 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1402/2849] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:47 2022] DELETE /identity/v3/users/88aa66a5c2fb47538f208ce878737a1c => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:48.754034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:48.755544 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:48.769212 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5d5e-edef-4fb2-ae6c-9d1a5252cf9a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:48.832748 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:48.833623 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:48.833904 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:48.834190 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] PATH_INFO: `/v3/projects/8682d34596554078a1eb4f0a561151cc` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:48.835373 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8682d34596554078a1eb4f0a561151cc)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:48.837552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:48.941671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8682d34596554078a1eb4f0a561151cc', 'request_id': 'req-146fcbde-152c-45b3-8fb1-a96e5c27ceba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:48.943749 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:48.948502 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8682d34596554078a1eb4f0a561151cc', 'request_id': 'req-146fcbde-152c-45b3-8fb1-a96e5c27ceba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:48.954100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8682d34596554078a1eb4f0a561151cc', 'request_id': 'req-146fcbde-152c-45b3-8fb1-a96e5c27ceba'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:49.036670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] The token cache is being invalidate because project 8682d34596554078a1eb4f0a561151cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:49.037138 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:49.038241 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-146fcbde-152c-45b3-8fb1-a96e5c27ceba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:49.040713 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1448/2850] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:48 2022] DELETE /identity/v3/projects/8682d34596554078a1eb4f0a561151cc => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:49.045570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.046030 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.047889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.065083 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:49.067355 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:49.130076 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1403/2851] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:49.137046 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.137340 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.137621 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.156478 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:49.157732 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:49.166380 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1449/2852] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:49.170735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.182863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58ff70c-2e61-4317-a0bf-ca752276f58b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.187723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.188422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.188813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.189161 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.189975 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:15:49.190293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.192528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:49.204771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1404/2853] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:15:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:15:49.209217 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.218673 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c440b56-ea4d-4b66-90cc-c76ab321b151 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.223892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.224547 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.225285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.225285 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.225812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.227710 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:49.344772 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1450/2854] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:49 2022] POST /identity/v3/projects => generated 385 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:49.349768 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.360099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67bb24a-ae38-4b51-b963-af416fdf2d58 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.405099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:49.427604 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.428441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.428863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.429328 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.430163 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.432402 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:49.526649 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1405/2855] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:49 2022] POST /identity/v3/users => generated 293 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:49.528342 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.540008 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683f5253-a766-436c-b1b0-ad8e4e82cd68 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.544967 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.545665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.545942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.546218 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.546957 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.548560 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:49.561666 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1451/2856] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:49.566629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.577032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a748915d-5846-490b-82d6-bea3b115edaa admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.582592 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aba262c-0af5-4364-995d-21e292e45a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.583339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba262c-0af5-4364-995d-21e292e45a1a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.583629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba262c-0af5-4364-995d-21e292e45a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.583904 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aba262c-0af5-4364-995d-21e292e45a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.584884 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba262c-0af5-4364-995d-21e292e45a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.586520 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aba262c-0af5-4364-995d-21e292e45a1a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:49.598125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1406/2857] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:49.604115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.616699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ab1a63-8375-4455-b99f-dbaa2bd49cd2 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.621542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.622668 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.623161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.623551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] PATH_INFO: `/v3/projects/a6f5180ca9494b18ad58ed8f95d1aec4/users/c3c9a33179114150bdb251974b8405e6/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.642610 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6f5180ca9494b18ad58ed8f95d1aec4, user_id=c3c9a33179114150bdb251974b8405e6, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.645338 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:49.735113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1452/2858] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:49 2022] PUT /identity/v3/projects/a6f5180ca9494b18ad58ed8f95d1aec4/users/c3c9a33179114150bdb251974b8405e6/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:49.749073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.749371 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.749826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.778885 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] MFA Rules not processed for user `c3c9a33179114150bdb251974b8405e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:49.780322 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:49.851919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1407/2859] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:49.856379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:49.921189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dbadc82-b478-4b0d-8c2a-7fa6283d914f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:49.925950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:49.926607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:49.927279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:49.927279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:49.927843 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:49.929769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.099827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1453/2860] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:49 2022] POST /identity/v3/projects => generated 387 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:50.101258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:50.113295 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8323f1-bebd-4b7a-bbd9-3b0841b6156f None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:50.172688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:50.173364 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.173641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.173915 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.174601 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:50.177133 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.298869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1408/2861] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:50 2022] POST /identity/v3/users => generated 300 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:50.304120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:50.321530 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8125e973-502b-4c19-8aa3-5600f99396fe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:50.327176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:50.327974 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.328255 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.328528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.329316 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:50.331660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.345025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1454/2862] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:50.350104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:50.364675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81d285fe-e157-4896-b1b7-65756bc067b2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:50.367564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:50.368551 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.368953 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.369291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] PATH_INFO: `/v3/projects/63d0377162ac47a8a2a207769258fc31/users/23ab6085e5214dedb700e4a226267dd9/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.391403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d0377162ac47a8a2a207769258fc31, user_id=23ab6085e5214dedb700e4a226267dd9, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:50.396728 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.492352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1409/2863] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:50 2022] PUT /identity/v3/projects/63d0377162ac47a8a2a207769258fc31/users/23ab6085e5214dedb700e4a226267dd9/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:50.496523 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:50.510245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a7be49-e4ed-4520-970c-45920159e6aa admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:50.589744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:50.603128 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68b7571-1edd-445d-9c1e-07214eb6ba39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:50.604492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b7571-1edd-445d-9c1e-07214eb6ba39 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.604879 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b7571-1edd-445d-9c1e-07214eb6ba39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.605196 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68b7571-1edd-445d-9c1e-07214eb6ba39 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.605976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b7571-1edd-445d-9c1e-07214eb6ba39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:50.607597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68b7571-1edd-445d-9c1e-07214eb6ba39 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.624986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1455/2864] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:50 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:50.626233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:50.639801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6c83a0-31c5-471b-8cb0-488b68bd49c2 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:50.645383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:50.646564 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.647164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.647626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] PATH_INFO: `/v3/projects/63d0377162ac47a8a2a207769258fc31/users/23ab6085e5214dedb700e4a226267dd9/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.649839 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] RBAC: Authorizing `identity:create_grant(project_id=63d0377162ac47a8a2a207769258fc31, user_id=23ab6085e5214dedb700e4a226267dd9, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:50.653717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.662866 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:50.669992 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1410/2865] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:50 2022] PUT /identity/v3/projects/63d0377162ac47a8a2a207769258fc31/users/23ab6085e5214dedb700e4a226267dd9/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:50.682324 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.683162 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.685009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.729773 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] MFA Rules not processed for user `23ab6085e5214dedb700e4a226267dd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:50.731009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:50.806751 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1456/2866] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:50 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:50.818628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:50.820367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:50.892974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873db7c-f146-4812-b006-b866fdadd158 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:50.898171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92fe10b3-31b0-4bf5-a930-0705761d634e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:50.898962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe10b3-31b0-4bf5-a930-0705761d634e service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:50.899356 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe10b3-31b0-4bf5-a930-0705761d634e service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:50.899538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92fe10b3-31b0-4bf5-a930-0705761d634e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:50.917687 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe10b3-31b0-4bf5-a930-0705761d634e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:50.919189 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92fe10b3-31b0-4bf5-a930-0705761d634e service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:50.930160 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1411/2867] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:50 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:52.151670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:52.230027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d63845-f3d5-4a3c-9d68-ad4fcf562309 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:52.234693 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:52.235583 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:52.235871 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:52.236154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] PATH_INFO: `/v3/users/c3c9a33179114150bdb251974b8405e6` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:52.237393 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3c9a33179114150bdb251974b8405e6)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:52.240012 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:52.394852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3c9a33179114150bdb251974b8405e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:52.395600 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3c9a33179114150bdb251974b8405e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:52.396173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3c9a33179114150bdb251974b8405e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:52.408044 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3c9a33179114150bdb251974b8405e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:52.411314 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:52.416660 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3c9a33179114150bdb251974b8405e6'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:52.494609 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1457/2868] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:52 2022] DELETE /identity/v3/users/c3c9a33179114150bdb251974b8405e6 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:52.499270 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:52.499747 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:52.499863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:52.517199 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:52.518430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:52.577530 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1412/2869] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:52.587892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:52.589578 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:52.648890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd877f43-9ac3-4124-8c1c-572ad1410003 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:52.652485 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:52.653161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:52.653448 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:52.653721 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:52.662796 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:52.664232 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:52.673635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1458/2870] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:52.925722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:52.936062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e973f-44a9-45f1-a4d3-3e16e9de8fbe None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:52.939901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:52.941176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:52.941464 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:52.941744 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:52.943753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:53.032785 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:53.034169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:53.061339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1413/2871] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:53.114456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:53.117507 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:53.154827 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a94ec0-f1a4-47b7-8d55-7e24f2cee1bd service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:53.160315 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:53.161264 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:53.161551 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:53.161831 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] PATH_INFO: `/v3/projects/a6f5180ca9494b18ad58ed8f95d1aec4` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:53.163072 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6f5180ca9494b18ad58ed8f95d1aec4)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:53.165113 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:53.256428 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6f5180ca9494b18ad58ed8f95d1aec4', 'request_id': 'req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.265053 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6f5180ca9494b18ad58ed8f95d1aec4', 'request_id': 'req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.271311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6f5180ca9494b18ad58ed8f95d1aec4', 'request_id': 'req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.346564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] The token cache is being invalidate because project a6f5180ca9494b18ad58ed8f95d1aec4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:53.346775 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.348489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.350098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1459/2872] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:53 2022] DELETE /identity/v3/projects/a6f5180ca9494b18ad58ed8f95d1aec4 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:53.354396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:53.359704 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:53.369168 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8c36b4-56cd-4566-be61-2d282a66c13d service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:53.427746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:53.428646 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:53.428932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:53.429207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] PATH_INFO: `/v3/users/23ab6085e5214dedb700e4a226267dd9` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:53.430431 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ab6085e5214dedb700e4a226267dd9)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:53.432387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:53.661672 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ab6085e5214dedb700e4a226267dd9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.674697 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ab6085e5214dedb700e4a226267dd9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.698159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ab6085e5214dedb700e4a226267dd9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.699303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ab6085e5214dedb700e4a226267dd9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.786179 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a6b5ae86-4faf-4af7-9128-5df637f40968 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ab6085e5214dedb700e4a226267dd9'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:53.796304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1414/2873] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:53 2022] DELETE /identity/v3/users/23ab6085e5214dedb700e4a226267dd9 => generated 0 bytes in 443 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:54.295882 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:54.296819 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:54.311851 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8cc477-ef3f-4962-aa50-3b14fd017fb1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:54.385173 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:54.395522 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:54.396444 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:54.396761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:54.397037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] PATH_INFO: `/v3/projects/63d0377162ac47a8a2a207769258fc31` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:54.398442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] RBAC: Authorizing `identity:delete_project(project_id=63d0377162ac47a8a2a207769258fc31)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:54.400372 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:54.531694 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '63d0377162ac47a8a2a207769258fc31', 'request_id': 'req-f4c39539-b1c1-4daa-812e-cca7f540208b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:54.538886 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '63d0377162ac47a8a2a207769258fc31', 'request_id': 'req-f4c39539-b1c1-4daa-812e-cca7f540208b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:54.544371 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '63d0377162ac47a8a2a207769258fc31', 'request_id': 'req-f4c39539-b1c1-4daa-812e-cca7f540208b'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:54.631604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] The token cache is being invalidate because project 63d0377162ac47a8a2a207769258fc31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:54.631877 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:54.632531 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-f4c39539-b1c1-4daa-812e-cca7f540208b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:54.634150 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1460/2874] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:54 2022] DELETE /identity/v3/projects/63d0377162ac47a8a2a207769258fc31 => generated 0 bytes in 340 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:54.639143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:54.639435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:54.639713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:54.660981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:54.662213 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:54.725906 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1415/2875] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:54.733863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:54.734157 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:54.734437 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:54.754868 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:54.756148 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:54.764394 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1461/2876] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:54.771074 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:54.782019 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb6fd67-746a-4bf3-b1fc-3c01cd433706 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:54.787708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:54.788379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:54.788786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:54.789108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] PATH_INFO: `/v3/domains` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:54.789824 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80395) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:15:54.790130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:54.791748 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:54.804208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1416/2877] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:15:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:15:54.808643 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:54.818787 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d78a73-d14c-4b0d-adac-6b2aeeb01015 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:54.823943 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:54.824808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:54.825221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:54.825588 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:54.826425 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:54.828421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:54.970755 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1462/2878] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:54 2022] POST /identity/v3/projects => generated 377 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:54.975679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:54.982166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:54.986147 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ce7ab7-9b74-4266-8d13-fc3a27039992 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.049150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.049752 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.050612 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.050958 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.051998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.055696 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.164591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1417/2879] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:54 2022] POST /identity/v3/users => generated 289 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:55.169259 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.180614 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34073e21-04b0-4435-b38f-1065c84457fe admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.185931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.186714 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.187009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.187302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.192054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.192054 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.203747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1463/2880] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:55.211375 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.223009 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5175e6cf-787b-442f-a8e3-13777612f5c3 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.228003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d29edcf8-cc67-4d21-b80e-835334f3707b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.228732 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edcf8-cc67-4d21-b80e-835334f3707b admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.229048 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edcf8-cc67-4d21-b80e-835334f3707b admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.229338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d29edcf8-cc67-4d21-b80e-835334f3707b admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.230078 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29edcf8-cc67-4d21-b80e-835334f3707b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.231722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d29edcf8-cc67-4d21-b80e-835334f3707b admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.242835 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1418/2881] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:55.247189 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.259441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee09e9b6-b223-4238-ba93-f9031100b42f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.264165 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.265129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.265414 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.265689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] PATH_INFO: `/v3/projects/a114c78796974885b87d67bb27d50419/users/0b2fa08795534a2a82f6c2ad52d148be/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.282813 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=a114c78796974885b87d67bb27d50419, user_id=0b2fa08795534a2a82f6c2ad52d148be, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.285596 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.369792 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1464/2882] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:55 2022] PUT /identity/v3/projects/a114c78796974885b87d67bb27d50419/users/0b2fa08795534a2a82f6c2ad52d148be/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:55.382384 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.382881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.383278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.414229 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] MFA Rules not processed for user `0b2fa08795534a2a82f6c2ad52d148be`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:55.415863 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:55.489415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1419/2883] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:55 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:55.494080 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.573782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f44e0d1a-7a76-49e4-acf9-a5efc557a7bc admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.578696 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.579378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.579663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.579942 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] PATH_INFO: `/v3/projects` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.580678 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.584118 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.719604 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1465/2884] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:55 2022] POST /identity/v3/projects => generated 381 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:55.724418 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.736097 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141562aa-1493-4822-a416-0ca91de304a1 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.797280 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.797993 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.798276 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.798549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] PATH_INFO: `/v3/users` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.799267 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.801873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.906879 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1420/2885] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:55 2022] POST /identity/v3/users => generated 297 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:55.914624 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.922433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:55.927263 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9361c6e8-3c9c-479e-9980-ce1703de4284 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.932986 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.933724 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.934006 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.934280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.935009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.936800 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:55.948951 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1466/2886] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:55.953274 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:55.959562 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:55.965171 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cd209e-a477-4094-bffe-9fe680c1e950 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:55.970080 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:55.970901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:55.971181 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:55.971455 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] PATH_INFO: `/v3/projects/33757350dd3e45e183d6f5cc2604690b/users/681f6e3a409a4c559d6388eb8df90aca/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:55.988006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33757350dd3e45e183d6f5cc2604690b, user_id=681f6e3a409a4c559d6388eb8df90aca, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:55.990813 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:56.095956 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1421/2887] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:55 2022] PUT /identity/v3/projects/33757350dd3e45e183d6f5cc2604690b/users/681f6e3a409a4c559d6388eb8df90aca/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:56.100505 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:56.111124 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a49d3cd1-2ad9-401c-88c1-b1031b40f88e admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:56.171759 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ae0fbd-c159-46d1-9e3b-b4640d7ee030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:56.172797 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0fbd-c159-46d1-9e3b-b4640d7ee030 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:56.173396 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0fbd-c159-46d1-9e3b-b4640d7ee030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:56.173712 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ae0fbd-c159-46d1-9e3b-b4640d7ee030 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:56.174463 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae0fbd-c159-46d1-9e3b-b4640d7ee030 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:56.176747 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ae0fbd-c159-46d1-9e3b-b4640d7ee030 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:56.188249 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1467/2888] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:15:56 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:15:56.193487 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:56.205107 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0372977d-fbe5-46eb-9e7b-40536bfa5968 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:56.211524 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:56.212498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:56.212820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:56.213098 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] PATH_INFO: `/v3/projects/33757350dd3e45e183d6f5cc2604690b/users/681f6e3a409a4c559d6388eb8df90aca/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:56.215100 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33757350dd3e45e183d6f5cc2604690b, user_id=681f6e3a409a4c559d6388eb8df90aca, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:56.218062 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:56.230778 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1422/2889] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:15:56 2022] PUT /identity/v3/projects/33757350dd3e45e183d6f5cc2604690b/users/681f6e3a409a4c559d6388eb8df90aca/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:56.242464 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:56.242880 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:56.243330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:56.273288 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] MFA Rules not processed for user `681f6e3a409a4c559d6388eb8df90aca`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:56.274528 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:56.364312 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1468/2890] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:56 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:56.375228 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:56.377028 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:56.464003 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bbebd-6d49-4b6e-ade2-9897285f3526 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:56.467846 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03afc6ae-6382-4fa7-a57e-d22c5890090a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:56.468506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc6ae-6382-4fa7-a57e-d22c5890090a service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:56.468820 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc6ae-6382-4fa7-a57e-d22c5890090a service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:56.469099 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03afc6ae-6382-4fa7-a57e-d22c5890090a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:56.479924 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afc6ae-6382-4fa7-a57e-d22c5890090a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:56.481549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03afc6ae-6382-4fa7-a57e-d22c5890090a service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:56.492881 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1423/2891] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:56 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:57.086840 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:57.151816 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-993af639-ca7d-447e-9928-7b6d6d5f4de5 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:57.156441 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:57.157383 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:57.157671 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:57.157947 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] PATH_INFO: `/v3/users/0b2fa08795534a2a82f6c2ad52d148be` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:57.159821 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b2fa08795534a2a82f6c2ad52d148be)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:57.162172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:57.182204 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:57.423260 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b2fa08795534a2a82f6c2ad52d148be'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:57.423921 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b2fa08795534a2a82f6c2ad52d148be'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:57.424533 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b2fa08795534a2a82f6c2ad52d148be'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:57.437145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b2fa08795534a2a82f6c2ad52d148be'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:57.446976 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b2fa08795534a2a82f6c2ad52d148be'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:57.552452 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1469/2892] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:57 2022] DELETE /identity/v3/users/0b2fa08795534a2a82f6c2ad52d148be => generated 0 bytes in 466 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:57.557079 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:57.557555 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:57.557647 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:57.575408 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:57.576644 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:57.651712 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1424/2893] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:57.662896 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:57.664434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:57.724549 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef2a171-89d6-4374-8258-878d9c690939 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:57.728115 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:57.728818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:57.729098 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:57.729376 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:57.741079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:57.742536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:57.752586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1470/2894] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:15:58.216108 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:58.217597 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:58.232890 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec63fe5-9cb5-4134-973e-d4b9a8a1b53c None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:58.238857 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:58.239771 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:58.240053 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:58.240397 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] PATH_INFO: `/v3/projects/a114c78796974885b87d67bb27d50419` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:58.242011 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a114c78796974885b87d67bb27d50419)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:58.243951 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:58.344242 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:58.347207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a114c78796974885b87d67bb27d50419', 'request_id': 'req-ea0ade14-423d-4a32-8e7e-bf283c464765'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.353694 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a114c78796974885b87d67bb27d50419', 'request_id': 'req-ea0ade14-423d-4a32-8e7e-bf283c464765'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.359052 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a114c78796974885b87d67bb27d50419', 'request_id': 'req-ea0ade14-423d-4a32-8e7e-bf283c464765'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.443880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] The token cache is being invalidate because project a114c78796974885b87d67bb27d50419 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:58.444114 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.444736 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.446537 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1425/2895] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:58 2022] DELETE /identity/v3/projects/a114c78796974885b87d67bb27d50419 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:58.450575 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:58.461155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73cc623-be93-4364-9cd8-072d61772ab8 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:58.519892 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:58.520814 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:58.521105 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:58.521384 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] PATH_INFO: `/v3/users/681f6e3a409a4c559d6388eb8df90aca` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:58.522635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] RBAC: Authorizing `identity:delete_user(user_id=681f6e3a409a4c559d6388eb8df90aca)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:58.524635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:58.681151 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=14,keystone:SELECT=85,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:15:58.688245 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '681f6e3a409a4c559d6388eb8df90aca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.689229 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '681f6e3a409a4c559d6388eb8df90aca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.690055 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '681f6e3a409a4c559d6388eb8df90aca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.703607 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '681f6e3a409a4c559d6388eb8df90aca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.712597 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-a112251a-ea3d-4cd3-9662-8d5cedb64aef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '681f6e3a409a4c559d6388eb8df90aca'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:58.814884 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1471/2896] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:15:58 2022] DELETE /identity/v3/users/681f6e3a409a4c559d6388eb8df90aca => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:59.273082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:59.274609 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:59.288001 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0ade14-423d-4a32-8e7e-bf283c464765 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:59.349193 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:59.350116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:59.350400 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:59.350686 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] PATH_INFO: `/v3/projects/33757350dd3e45e183d6f5cc2604690b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:59.351875 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33757350dd3e45e183d6f5cc2604690b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:59.353832 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:59.499396 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33757350dd3e45e183d6f5cc2604690b', 'request_id': 'req-f1540eb4-b38f-49b0-8193-5cc17682b5c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:59.506648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33757350dd3e45e183d6f5cc2604690b', 'request_id': 'req-f1540eb4-b38f-49b0-8193-5cc17682b5c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:59.512278 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33757350dd3e45e183d6f5cc2604690b', 'request_id': 'req-f1540eb4-b38f-49b0-8193-5cc17682b5c6'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:59.608442 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] The token cache is being invalidate because project 33757350dd3e45e183d6f5cc2604690b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:15:59.608757 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:59.609377 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-f1540eb4-b38f-49b0-8193-5cc17682b5c6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:15:59.610941 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1426/2897] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:15:59 2022] DELETE /identity/v3/projects/33757350dd3e45e183d6f5cc2604690b => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:15:59.616761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:59.617258 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:59.617378 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:59.636764 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:59.637991 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:59.702488 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1472/2898] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:59.711306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:59.711591 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:59.711867 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:59.731427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:15:59.732713 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:15:59.741176 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1427/2899] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:15:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:15:59.746493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:59.756959 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f11e17-be58-4e09-998b-53324ac8ba1d None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:59.761592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:59.762261 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:59.762568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:59.762849 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:59.763594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:15:59.763899 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:59.765524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:59.779563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1473/2900] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:15:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:15:59.784233 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:59.794593 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd29826-2ea1-47e7-b27d-43b6a7878cee None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:59.799404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:15:59.800663 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:15:59.800950 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:15:59.801221 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:15:59.801943 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:15:59.803936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:15:59.936889 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1428/2901] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:15:59 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:15:59.941707 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:59.952221 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3d846f-22fc-4b08-93b6-042215a1cad8 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:15:59.992090 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:15:59.994401 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:00.012912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.019534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.019534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.019534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.019534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.019534 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.038483 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:00.082589 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f193023-acec-4737-8874-68b3cb3bfc7f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.088104 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.088795 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.089084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.089362 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.091286 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:00.103391 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.105026 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.117116 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1429/2902] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:15:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:16:00.130313 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1474/2903] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:15:59 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:16:00.134622 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.145746 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2236ea32-ac7f-484d-9e03-6da5fbf7f018 service placement] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.150549 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.151217 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.151498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.151920 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.152684 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.154301 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.165222 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1430/2904] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:16:00.169506 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.181373 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c38dac-8f35-4d2c-bda0-98b76c9df262 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.191480 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c454db-c218-4928-a991-01d3e327f73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.192237 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c454db-c218-4928-a991-01d3e327f73a admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.192608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c454db-c218-4928-a991-01d3e327f73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.192823 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c454db-c218-4928-a991-01d3e327f73a admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.193564 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c454db-c218-4928-a991-01d3e327f73a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.195178 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c454db-c218-4928-a991-01d3e327f73a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.207292 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1475/2905] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:16:00.211349 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.221514 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1daa6c-e731-4541-897b-991781293386 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.226998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.227895 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.228182 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.228466 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] PATH_INFO: `/v3/projects/22c0d57ff8374a6189f252be1356f97e/users/64e25c85292a41eaa01029897b8a706d/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.245797 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=22c0d57ff8374a6189f252be1356f97e, user_id=64e25c85292a41eaa01029897b8a706d, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.248502 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.340639 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1431/2906] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:16:00 2022] PUT /identity/v3/projects/22c0d57ff8374a6189f252be1356f97e/users/64e25c85292a41eaa01029897b8a706d/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:00.352545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.352912 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.353220 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.381009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] MFA Rules not processed for user `64e25c85292a41eaa01029897b8a706d`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:00.382302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:00.393197 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:00.458857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1476/2907] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:00 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:00.459826 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.526143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd81b827-3d1f-40f7-9195-ee7c11c3dd8e admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.531339 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.531959 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.532249 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.532528 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.533584 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.536092 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.688480 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1432/2908] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:16:00 2022] POST /identity/v3/projects => generated 383 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:16:00.689876 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.701585 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8774919a-9c01-4c15-b65e-eb46be881b2f None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.764123 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.764857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.765145 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.765422 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.766121 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.768143 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.857160 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1477/2909] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:16:00 2022] POST /identity/v3/users => generated 298 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:16:00.858605 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.870335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c45df40-365a-4e25-baf2-db0225ef42bc admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.878981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.878981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.878981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.878981 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.879345 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.880321 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:00.891731 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1433/2910] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 23 14:16:00.896329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:00.907000 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1e54a0-9b14-4340-822c-b0a975bcbeb4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:00.913199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:00.914161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:00.914456 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:00.914736 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] PATH_INFO: `/v3/projects/287078af25924059a54c717dabf7946a/users/16746ecc63b84d81991416ebe1f70e00/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:00.930744 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] RBAC: Authorizing `identity:create_grant(project_id=287078af25924059a54c717dabf7946a, user_id=16746ecc63b84d81991416ebe1f70e00, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:00.933493 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:01.026684 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1478/2911] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:16:00 2022] PUT /identity/v3/projects/287078af25924059a54c717dabf7946a/users/16746ecc63b84d81991416ebe1f70e00/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:01.030873 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:01.041786 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8e33f-8dd4-47d5-80fd-cb53f3471b15 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:01.094596 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:01.102293 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8d7a16-8bbc-4337-b6b0-26534c405b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:01.104350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7a16-8bbc-4337-b6b0-26534c405b3a admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:01.104350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7a16-8bbc-4337-b6b0-26534c405b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:01.104350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8d7a16-8bbc-4337-b6b0-26534c405b3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:01.104350 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d7a16-8bbc-4337-b6b0-26534c405b3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:01.105902 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8d7a16-8bbc-4337-b6b0-26534c405b3a admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:01.117253 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1434/2912] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:16:01.121562 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:01.132536 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b4957d-9e33-4d3a-a255-f18c5989287a admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:01.137434 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:01.138329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] REQUEST_METHOD: `PUT` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:01.138663 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:01.138965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] PATH_INFO: `/v3/projects/287078af25924059a54c717dabf7946a/users/16746ecc63b84d81991416ebe1f70e00/roles/33de09388f974cd391e13bb4f2502a52` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:01.141091 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=287078af25924059a54c717dabf7946a, user_id=16746ecc63b84d81991416ebe1f70e00, role_id=33de09388f974cd391e13bb4f2502a52)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:01.143711 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:01.154563 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1479/2913] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:16:01 2022] PUT /identity/v3/projects/287078af25924059a54c717dabf7946a/users/16746ecc63b84d81991416ebe1f70e00/roles/33de09388f974cd391e13bb4f2502a52 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:01.166036 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:01.166347 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:01.166642 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:01.196670 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] MFA Rules not processed for user `16746ecc63b84d81991416ebe1f70e00`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:01.198004 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:01.267582 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1435/2914] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:01.279820 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:01.281525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:01.341586 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18d1a709-3c2d-4671-88e3-d40bfcd819ee admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:01.345385 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:01.346043 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:01.346330 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:01.347985 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:01.368256 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:01.414449 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:01.415930 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:01.426305 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1480/2915] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:02.991869 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:02.993306 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:03.082799 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cab5d2-6131-4eff-bc07-a432987b53a4 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:03.089040 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:03.090730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:03.090730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:03.090730 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:03.093075 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:03.107882 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:03.109534 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:03.129414 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1436/2916] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:07.864093 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:07.872998 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bbeb8c-eb9f-4d79-83fd-ee8ccb7e2a9a service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:07.879442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:07.879442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:07.879442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:07.879442 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:07.887697 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:07.889211 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:07.908328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1481/2917] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:08.823143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:08.824928 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:08.842781 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76c39e2-deca-4483-8882-59f57075a81c service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:08.860701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:08.860701 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:08.860923 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:08.861318 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:08.864415 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:08.878961 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:08.889207 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcfbb947-9646-47bf-b66f-bc0fab45a2cd service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:08.894314 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1437/2918] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:14.885415 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:14.951897 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d3687d-1362-4558-b9c9-b0996819cae5 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:14.957659 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:14.958558 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:14.958838 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:14.959106 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] PATH_INFO: `/v3/users/64e25c85292a41eaa01029897b8a706d` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:14.960525 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=64e25c85292a41eaa01029897b8a706d)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:14.962526 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:15.155966 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64e25c85292a41eaa01029897b8a706d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:15.156665 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64e25c85292a41eaa01029897b8a706d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:15.157279 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64e25c85292a41eaa01029897b8a706d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:15.170364 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64e25c85292a41eaa01029897b8a706d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:15.178983 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64e25c85292a41eaa01029897b8a706d'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:15.273353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1482/2919] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:16:14 2022] DELETE /identity/v3/users/64e25c85292a41eaa01029897b8a706d => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:15.277878 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:15.278367 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:15.278457 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:15.300169 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:15.301452 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:15.348886 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=100 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:15.365304 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1438/2920] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:15.374960 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:15.376545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:15.405746 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:15.435497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f2137f-79c1-47e9-b4d3-f0d18a4a84d7 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:15.440081 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:15.440790 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:15.441230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:15.441517 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:15.453034 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:15.454494 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:15.472890 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1483/2921] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:15.931550 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:15.933387 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:15.948618 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16831777-e2b4-4c13-b314-71ae78483ad0 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:15.954142 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:15.955070 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:15.955332 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:15.955614 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] PATH_INFO: `/v3/projects/22c0d57ff8374a6189f252be1356f97e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:15.957936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22c0d57ff8374a6189f252be1356f97e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:15.962628 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:16.097669 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22c0d57ff8374a6189f252be1356f97e', 'request_id': 'req-7e6811b2-69ec-4863-aa64-3c0c60c370b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.103980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22c0d57ff8374a6189f252be1356f97e', 'request_id': 'req-7e6811b2-69ec-4863-aa64-3c0c60c370b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.109722 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22c0d57ff8374a6189f252be1356f97e', 'request_id': 'req-7e6811b2-69ec-4863-aa64-3c0c60c370b8'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.192441 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] The token cache is being invalidate because project 22c0d57ff8374a6189f252be1356f97e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:16:16.192763 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.193439 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.194983 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1439/2922] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:16:15 2022] DELETE /identity/v3/projects/22c0d57ff8374a6189f252be1356f97e => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:16.198931 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:16.209917 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64a6b80-2419-4f7e-ba3c-823d71f715f1 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:16.267568 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:16.269117 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:16.270063 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:16.270352 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] PATH_INFO: `/v3/users/16746ecc63b84d81991416ebe1f70e00` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:16.271608 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=16746ecc63b84d81991416ebe1f70e00)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:16.273598 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:16.477176 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16746ecc63b84d81991416ebe1f70e00'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.477934 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16746ecc63b84d81991416ebe1f70e00'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.478591 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16746ecc63b84d81991416ebe1f70e00'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.492652 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16746ecc63b84d81991416ebe1f70e00'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.500387 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-183a7dbc-c034-4cb6-8a61-6fd4f0de49bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16746ecc63b84d81991416ebe1f70e00'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:16.580086 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1484/2923] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:16:16 2022] DELETE /identity/v3/users/16746ecc63b84d81991416ebe1f70e00 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:17.028327 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.030225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:17.044914 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6811b2-69ec-4863-aa64-3c0c60c370b8 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.110807 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.112435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.113224 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.113346 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] PATH_INFO: `/v3/projects/287078af25924059a54c717dabf7946a` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.114575 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] RBAC: Authorizing `identity:delete_project(project_id=287078af25924059a54c717dabf7946a)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.116675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:17.262125 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '287078af25924059a54c717dabf7946a', 'request_id': 'req-572f3817-590c-4401-9707-c4b78b47f794'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:17.269662 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '287078af25924059a54c717dabf7946a', 'request_id': 'req-572f3817-590c-4401-9707-c4b78b47f794'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:17.275433 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '287078af25924059a54c717dabf7946a', 'request_id': 'req-572f3817-590c-4401-9707-c4b78b47f794'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:17.362979 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] The token cache is being invalidate because project 287078af25924059a54c717dabf7946a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:16:17.363303 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:17.363936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-572f3817-590c-4401-9707-c4b78b47f794 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:17.365510 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1440/2924] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:16:17 2022] DELETE /identity/v3/projects/287078af25924059a54c717dabf7946a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:17.371518 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.372185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.372185 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.392014 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:17.393317 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:17.436863 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:17.459357 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1485/2925] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:16:17.469919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.470214 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.470499 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.481422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:17.494469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:17.495719 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:17.504445 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1441/2926] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:17.509030 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.518680 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad721-0608-49b8-be26-7cbde400bfa6 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.526064 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.527114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.527545 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.527881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.530032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:16:17.530358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.532003 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:17.544537 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1486/2927] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:16:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:16:17.549150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.558787 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e229c3-744b-4220-bcd4-c8440edcc56b None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.564335 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.565602 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.565982 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.566357 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.567084 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.569063 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:17.691850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1442/2928] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:16:17 2022] POST /identity/v3/projects => generated 385 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:16:17.696780 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.708280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a726e9a1-c1a0-4924-9140-963cce554ccd admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.764236 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.765922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.765922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.765922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.766194 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.768129 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:17.867817 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1487/2929] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:16:17 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:16:17.872754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.884498 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c2b8d9-af5d-4c5a-943e-7a5e46d99f64 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.889240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.889892 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.890175 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.890523 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.891166 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.892901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:17.904240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1443/2930] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:16:17.908390 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.920047 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d609bb39-ca98-4245-9bc2-fa2c77e4f814 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.925345 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51efc0f3-6be8-47e7-b3f3-a49c240f0039 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.926027 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efc0f3-6be8-47e7-b3f3-a49c240f0039 admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.926311 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efc0f3-6be8-47e7-b3f3-a49c240f0039 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.926592 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51efc0f3-6be8-47e7-b3f3-a49c240f0039 admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.927333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efc0f3-6be8-47e7-b3f3-a49c240f0039 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.929069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51efc0f3-6be8-47e7-b3f3-a49c240f0039 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:17.942801 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1488/2931] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:16:17.948424 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:17.959735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f50c74-7f6c-4903-b1a3-5fd2d56fc821 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:17.964685 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:17.967880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:17.967880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:17.967880 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] PATH_INFO: `/v3/projects/7c0db9639dc34f76a59530b5539fe507/users/52485d4738694020b199c610c018c19c/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:17.982656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] RBAC: Authorizing `identity:create_grant(project_id=7c0db9639dc34f76a59530b5539fe507, user_id=52485d4738694020b199c610c018c19c, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:17.985989 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:18.089469 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1444/2932] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:16:17 2022] PUT /identity/v3/projects/7c0db9639dc34f76a59530b5539fe507/users/52485d4738694020b199c610c018c19c/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:18.101705 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:18.101996 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:18.102277 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:18.131374 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] MFA Rules not processed for user `52485d4738694020b199c610c018c19c`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:18.132524 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:18.199465 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1489/2933] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:18 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:18.211800 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:18.213577 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:18.282289 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1178002-0d67-4984-86ec-3a6f728e247f admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:18.286939 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7318a1-5175-46f4-b626-7be4afae2b89 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:18.287830 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7318a1-5175-46f4-b626-7be4afae2b89 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:18.288122 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7318a1-5175-46f4-b626-7be4afae2b89 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:18.288404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7318a1-5175-46f4-b626-7be4afae2b89 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:18.298029 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7318a1-5175-46f4-b626-7be4afae2b89 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:18.299491 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7318a1-5175-46f4-b626-7be4afae2b89 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:18.310936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1445/2934] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:28.196492 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=78,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:28.306369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:30.747301 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:30.814499 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d73c8-1b15-46ed-8413-705506f3a7f2 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:30.820230 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:30.821146 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:30.821462 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:30.821769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] PATH_INFO: `/v3/users/52485d4738694020b199c610c018c19c` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:30.822958 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] RBAC: Authorizing `identity:delete_user(user_id=52485d4738694020b199c610c018c19c)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:30.824908 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:31.018661 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52485d4738694020b199c610c018c19c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.019247 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52485d4738694020b199c610c018c19c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.019845 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52485d4738694020b199c610c018c19c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.032231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52485d4738694020b199c610c018c19c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.040137 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52485d4738694020b199c610c018c19c'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.117794 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1490/2935] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:16:30 2022] DELETE /identity/v3/users/52485d4738694020b199c610c018c19c => generated 0 bytes in 371 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:31.122652 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:31.123136 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:31.123240 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:31.141444 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:31.142671 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:31.205538 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1446/2936] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:31.215552 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:31.217037 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:31.277155 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6416aade-89ab-4133-9119-d1ceb4c20245 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:31.280631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91613dbe-7fca-4dab-93d1-06115c2af2de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:31.281280 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613dbe-7fca-4dab-93d1-06115c2af2de service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:31.281561 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613dbe-7fca-4dab-93d1-06115c2af2de service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:31.281836 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91613dbe-7fca-4dab-93d1-06115c2af2de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:31.291731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91613dbe-7fca-4dab-93d1-06115c2af2de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:31.293199 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91613dbe-7fca-4dab-93d1-06115c2af2de service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:31.302916 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1491/2937] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:31.770934 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:31.772494 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:31.785353 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd889afa-fc29-46fc-b2fe-8f7d51718cb2 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:31.790204 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:31.791082 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:31.791369 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:31.791648 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] PATH_INFO: `/v3/projects/7c0db9639dc34f76a59530b5539fe507` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:31.792874 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7c0db9639dc34f76a59530b5539fe507)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:31.794803 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:31.936105 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7c0db9639dc34f76a59530b5539fe507', 'request_id': 'req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.943427 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7c0db9639dc34f76a59530b5539fe507', 'request_id': 'req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:31.949027 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7c0db9639dc34f76a59530b5539fe507', 'request_id': 'req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:32.055411 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] The token cache is being invalidate because project 7c0db9639dc34f76a59530b5539fe507 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:16:32.055629 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:32.056422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-a9b62dda-6d61-4bd7-9f91-ffa1f0d5aa73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:16:32.058159 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1447/2938] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:16:31 2022] DELETE /identity/v3/projects/7c0db9639dc34f76a59530b5539fe507 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:32.063168 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.063638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.063757 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.083077 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:32.084303 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:32.102100 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:32.152841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1492/2939] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:16:32.161194 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.161753 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.161936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.182338 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:32.183570 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:32.194037 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1448/2940] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:32.199010 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.209079 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd77f2fa-1d68-4855-b98d-d84cc294a76b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.213734 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.214401 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.214687 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.214965 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] PATH_INFO: `/v3/domains` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.215699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80396) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 23 14:16:32.216009 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.218032 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.229781 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1493/2941] 10.222.0.70 () {58 vars in 1152 bytes} [Mon May 23 14:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 23 14:16:32.234225 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.244190 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ba110a-fc44-4937-8a39-fc7b855245db None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.248766 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.249422 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.249705 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.249980 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.250675 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.253405 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.337150 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1449/2942] 10.222.0.70 () {60 vars in 1152 bytes} [Mon May 23 14:16:32 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 23 14:16:32.341337 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.352322 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb4fe27-93a5-4c1e-8a71-f618244e6c4d admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.411636 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.412328 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.412644 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.412952 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] PATH_INFO: `/v3/users` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.413670 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.415622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.506803 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1494/2943] 10.222.0.70 () {60 vars in 1143 bytes} [Mon May 23 14:16:32 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 23 14:16:32.510997 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.522258 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73738857-11d5-4676-b9b9-0f1c9322f9fd admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.529637 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.530325 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.530613 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.530891 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.531630 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.533374 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.546478 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1450/2944] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:16:32.551395 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.561994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d17947-6d65-4fd0-8a3e-0c7f25a521f4 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.568240 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978efb8-2426-4df7-bdc9-8e031d35620b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.568972 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978efb8-2426-4df7-bdc9-8e031d35620b admin admin] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.569268 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978efb8-2426-4df7-bdc9-8e031d35620b admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.569542 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978efb8-2426-4df7-bdc9-8e031d35620b admin admin] PATH_INFO: `/v3/roles` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.570283 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978efb8-2426-4df7-bdc9-8e031d35620b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.571900 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e978efb8-2426-4df7-bdc9-8e031d35620b admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.584078 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1495/2945] 10.222.0.70 () {58 vars in 1121 bytes} [Mon May 23 14:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 23 14:16:32.588311 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.598972 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a2031d-31ef-4052-989b-56fb072d4fa5 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.603525 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.604390 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] REQUEST_METHOD: `PUT` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.604700 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.604974 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] PATH_INFO: `/v3/projects/fe1d53c6f92c4f758d923117a5350a8b/users/3aac1d6fc134477083573a0ffeb16d1a/roles/d5653c8adc4e4a88a006171b36c6db7e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.621067 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe1d53c6f92c4f758d923117a5350a8b, user_id=3aac1d6fc134477083573a0ffeb16d1a, role_id=d5653c8adc4e4a88a006171b36c6db7e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.623735 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.729430 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1451/2946] 10.222.0.70 () {60 vars in 1482 bytes} [Mon May 23 14:16:32 2022] PUT /identity/v3/projects/fe1d53c6f92c4f758d923117a5350a8b/users/3aac1d6fc134477083573a0ffeb16d1a/roles/d5653c8adc4e4a88a006171b36c6db7e => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:16:32.741114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.741403 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.741681 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.770752 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] MFA Rules not processed for user `3aac1d6fc134477083573a0ffeb16d1a`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:16:32.772015 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:32.843433 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1496/2947] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:16:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:16:32.854688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:16:32.856404 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:16:32.880145 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:32.915031 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa77db40-9291-4093-9d30-92a4778d0cea admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:16:32.918912 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df58580-9bc9-4445-9b01-ec5e332cdbd1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:16:32.919569 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df58580-9bc9-4445-9b01-ec5e332cdbd1 service cinder] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:16:32.919903 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df58580-9bc9-4445-9b01-ec5e332cdbd1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:16:32.920164 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df58580-9bc9-4445-9b01-ec5e332cdbd1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:16:32.930326 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df58580-9bc9-4445-9b01-ec5e332cdbd1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:16:32.931780 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df58580-9bc9-4445-9b01-ec5e332cdbd1 service cinder] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:16:32.942155 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1452/2948] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:16:32 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:16:42.839001 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:16:43.059840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=17 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:17:03.673631 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:17:03.738251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf46d0-1036-4346-bcbc-c13db0400822 None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:17:03.744075 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:17:03.745035 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:17:03.745321 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:17:03.745601 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] PATH_INFO: `/v3/users/3aac1d6fc134477083573a0ffeb16d1a` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:17:03.746812 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3aac1d6fc134477083573a0ffeb16d1a)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:17:03.748885 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:17:03.948134 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3aac1d6fc134477083573a0ffeb16d1a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:03.948799 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3aac1d6fc134477083573a0ffeb16d1a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:03.951062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3aac1d6fc134477083573a0ffeb16d1a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:03.962392 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3aac1d6fc134477083573a0ffeb16d1a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:03.970262 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3aac1d6fc134477083573a0ffeb16d1a'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:04.077620 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1497/2949] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:17:03 2022] DELETE /identity/v3/users/3aac1d6fc134477083573a0ffeb16d1a => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:17:04.082203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:17:04.082717 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:17:04.082828 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:17:04.111965 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] MFA Rules not processed for user `5acd460bee394e719cdafd0e39f18395`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:17:04.113208 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:17:04.173042 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1453/2950] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 23 14:17:04.183347 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:17:04.184815 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:17:04.243638 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e92c4d-1839-4d7f-bb51-7e896d33a1b1 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:17:04.247172 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:17:04.247829 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:17:04.248114 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:17:04.248397 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:17:04.257346 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:17:04.258774 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:17:04.268069 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1498/2951] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:17:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:17:04.765366 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:17:04.767265 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:17:04.781706 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b26449-0f9b-4494-acc6-f870ccbd34bf None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:17:04.786708 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:17:04.787585 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:17:04.787860 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:17:04.788137 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] PATH_INFO: `/v3/projects/fe1d53c6f92c4f758d923117a5350a8b` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:17:04.789383 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe1d53c6f92c4f758d923117a5350a8b)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:17:04.791450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:17:04.962148 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe1d53c6f92c4f758d923117a5350a8b', 'request_id': 'req-3e2a6c12-4871-4c37-9970-31c3b9f87174'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:04.969091 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe1d53c6f92c4f758d923117a5350a8b', 'request_id': 'req-3e2a6c12-4871-4c37-9970-31c3b9f87174'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:04.975308 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe1d53c6f92c4f758d923117a5350a8b', 'request_id': 'req-3e2a6c12-4871-4c37-9970-31c3b9f87174'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:05.060656 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] The token cache is being invalidate because project fe1d53c6f92c4f758d923117a5350a8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80395) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:17:05.061043 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:05.061854 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-3e2a6c12-4871-4c37-9970-31c3b9f87174 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:17:05.063948 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1454/2952] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:17:04 2022] DELETE /identity/v3/projects/fe1d53c6f92c4f758d923117a5350a8b => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:17:14.263940 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:17:15.057006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=58,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:18:11.554594 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:18:11.556025 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:11.614209 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed391e9-d26c-442b-ac23-7d1187fd41d7 service neutron] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:18:11.666982 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:18:11.667639 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:18:11.667925 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:18:11.668201 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:18:11.670140 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:11.729339 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:18:11.730848 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:18:11.739841 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1499/2953] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:18:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:18:21.735635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=68 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:18:22.305241 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] REQUEST_METHOD: `POST` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:18:22.305754 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:18:22.305853 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:18:22.326527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.auth.core [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] MFA Rules not processed for user `7b6534d42be64c3cb311d384458bfae3`. Rule list: `[]` (Enabled: `True`). {{(pid=80395) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:18:22.327739 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:22.334645 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1455/2954] 10.222.0.70 () {58 vars in 957 bytes} [Mon May 23 14:18:22 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:18:22.373808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:18:22.375231 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:22.386420 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-104a1bb7-3a97-4a1a-b71a-4e50102ba394 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:18:22.390761 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1ff9d52fc1194828898369d1237b1337', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:18:22.391416 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:18:22.391702 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:18:22.391984 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:18:22.393852 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:22.494379 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:18:22.495771 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:18:22.517062 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1500/2955] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:18:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:18:32.320660 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=3 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:18:32.514497 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=46 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:18:45.896506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:18:45.896506 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:45.971453 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9be6dc8-9c4a-48fc-9e50-9aacf2796a63 None None] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:18:45.975196 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:18:45.975850 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:18:45.976131 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:18:45.976407 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:18:45.978257 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:18:46.047668 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:18:46.049184 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:18:46.061143 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1456/2956] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:18:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:18:56.055140 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=59 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:19:11.663881 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:19:11.673471 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a644bcf7-9593-41ed-b99f-e5e428edd448 service cinder] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:19:11.677154 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07744ded-6950-481c-964d-b236847d850c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:19:11.677802 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07744ded-6950-481c-964d-b236847d850c service neutron] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:19:11.678087 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07744ded-6950-481c-964d-b236847d850c service neutron] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:19:11.678358 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07744ded-6950-481c-964d-b236847d850c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:19:11.680147 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-07744ded-6950-481c-964d-b236847d850c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:19:11.690994 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07744ded-6950-481c-964d-b236847d850c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:19:11.692417 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07744ded-6950-481c-964d-b236847d850c service neutron] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:19:11.700733 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1501/2957] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:19:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:19:21.697023 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=6 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:20:35.412412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:20:35.418840 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:35.431216 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c049c0b0-953c-4031-bd3f-bb7299be7aa1 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:20:35.440219 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9cba93afd3344df298ba353d9994ac37', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'reader', 'ResellerAdmin', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:20:35.441033 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:20:35.441379 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:20:35.441727 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:20:35.443641 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:35.458526 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:20:35.460412 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:20:35.473765 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1457/2958] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:20:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:20:45.468361 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=6 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:20:51.531856 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] REQUEST_METHOD: `POST` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:20:51.532404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:20:51.532404 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:20:51.591612 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.auth.core [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] MFA Rules not processed for user `4ea47f7631ce4094afd5c0c2787f8969`. Rule list: `[]` (Enabled: `True`). {{(pid=80396) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 23 14:20:51.592857 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:51.599481 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1502/2959] 10.222.0.70 () {58 vars in 1061 bytes} [Mon May 23 14:20:51 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 23 14:20:51.606919 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:20:51.616527 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1600ea7b-be38-48d6-91f2-7de74ec4642b service nova] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:20:51.620130 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4ea47f7631ce4094afd5c0c2787f8969', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:20:51.622045 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:20:51.622352 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:20:51.622632 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:20:51.624503 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:51.724446 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:20:51.725932 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:20:51.746625 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1458/2960] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:20:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:20:55.549635 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:20:55.551120 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:55.644477 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d001fe0-4ce4-4720-a201-8ad73254095b None None] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:20:55.649689 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:20:55.650355 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:20:55.650641 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:20:55.650922 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:20:55.652950 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:55.665362 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:20:55.670743 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:20:55.684699 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1503/2961] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:20:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:20:59.506962 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:20:59.508450 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:20:59.524482 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e615c52d-47ab-4962-a3f7-f202dc65e2a9 service neutron] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:20:59.528723 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:20:59.529707 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:20:59.529996 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:20:59.530281 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] PATH_INFO: `/v3/users/7b6534d42be64c3cb311d384458bfae3` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:20:59.531501 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b6534d42be64c3cb311d384458bfae3)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:20:59.533576 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:20:59.729793 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b6534d42be64c3cb311d384458bfae3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:20:59.736877 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b6534d42be64c3cb311d384458bfae3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:20:59.750998 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b6534d42be64c3cb311d384458bfae3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:20:59.751677 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b6534d42be64c3cb311d384458bfae3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:20:59.833203 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b6534d42be64c3cb311d384458bfae3'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:20:59.840761 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1459/2962] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:20:59 2022] DELETE /identity/v3/users/7b6534d42be64c3cb311d384458bfae3 => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:21:00.117333 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:21:00.130239 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e989fce9-5d9a-4e0c-ac8e-f06655d2097c service placement] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:21:00.192353 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:21:00.193271 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:21:00.193555 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:21:00.193832 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] PATH_INFO: `/v3/projects/aee15038cdf945a594437e83a34e1afe` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:21:00.195038 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee15038cdf945a594437e83a34e1afe)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:21:00.197028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:21:00.301692 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:21:00.305161 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee15038cdf945a594437e83a34e1afe', 'request_id': 'req-8248f6c2-b9ca-493b-98e5-1b0ab201718f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.311894 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee15038cdf945a594437e83a34e1afe', 'request_id': 'req-8248f6c2-b9ca-493b-98e5-1b0ab201718f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.317421 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee15038cdf945a594437e83a34e1afe', 'request_id': 'req-8248f6c2-b9ca-493b-98e5-1b0ab201718f'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.386782 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] The token cache is being invalidate because project aee15038cdf945a594437e83a34e1afe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:21:00.387170 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.387818 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.389511 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1504/2963] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:21:00 2022] DELETE /identity/v3/projects/aee15038cdf945a594437e83a34e1afe => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:21:00.393665 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:21:00.395087 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:21:00.410025 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe46a953-17cc-4398-a087-cfe2ade7ec40 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:21:00.455594 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:21:00.479893 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:21:00.482032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:21:00.482032 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:21:00.482363 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] PATH_INFO: `/v3/users/3268c3cede5e4635ab3cb9390c407c0e` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:21:00.483940 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3268c3cede5e4635ab3cb9390c407c0e)` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:21:00.485901 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:21:00.678936 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3268c3cede5e4635ab3cb9390c407c0e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.688462 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3268c3cede5e4635ab3cb9390c407c0e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.702435 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3268c3cede5e4635ab3cb9390c407c0e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.709688 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3268c3cede5e4635ab3cb9390c407c0e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.815802 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.notifications [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3268c3cede5e4635ab3cb9390c407c0e'} {{(pid=80395) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:00.822291 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1460/2964] 10.222.0.70 () {58 vars in 1223 bytes} [Mon May 23 14:21:00 2022] DELETE /identity/v3/users/3268c3cede5e4635ab3cb9390c407c0e => generated 0 bytes in 429 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:21:01.068042 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:21:01.069622 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:21:01.084028 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8248f6c2-b9ca-493b-98e5-1b0ab201718f admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:21:01.143859 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5acd460bee394e719cdafd0e39f18395', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8db400662d744ef19141527de1d2418c', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:21:01.144795 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:21:01.145082 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:21:01.145360 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] PATH_INFO: `/v3/projects/702a8f83e4db4e0296fcfadcedb6b5b1` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:21:01.146535 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=702a8f83e4db4e0296fcfadcedb6b5b1)` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:21:01.148489 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:21:01.236532 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '702a8f83e4db4e0296fcfadcedb6b5b1', 'request_id': 'req-6d34c853-afd0-42d6-88dc-89a8bb55e841'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:01.242739 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '702a8f83e4db4e0296fcfadcedb6b5b1', 'request_id': 'req-6d34c853-afd0-42d6-88dc-89a8bb55e841'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:01.248251 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '702a8f83e4db4e0296fcfadcedb6b5b1', 'request_id': 'req-6d34c853-afd0-42d6-88dc-89a8bb55e841'} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:01.306808 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] The token cache is being invalidate because project 702a8f83e4db4e0296fcfadcedb6b5b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80396) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 23 14:21:01.307096 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:01.307731 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.notifications [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80396) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 23 14:21:01.309470 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1505/2965] 10.222.0.70 () {58 vars in 1232 bytes} [Mon May 23 14:21:01 2022] DELETE /identity/v3/projects/702a8f83e4db4e0296fcfadcedb6b5b1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 23 14:21:06.982008 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Authenticating user token {{(pid=80395) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:21:06.984913 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:21:07.073792 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf09b89-6763-4560-9e5d-f30169df8516 admin admin] Validating token access rules against request {{(pid=80395) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:21:07.141801 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:21:07.149403 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80395) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:21:07.150073 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] REQUEST_METHOD: `GET` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:21:07.150354 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] SCRIPT_NAME: `/identity` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:21:07.150626 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80395) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:21:07.152737 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.fernet_utils [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80395) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:21:07.170006 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:21:07.171679 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef639cf1-c25e-4454-a83e-a1464d551127 service placement] RBAC: Authorization granted {{(pid=80395) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:21:07.186540 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: [pid: 80395|app: 0|req: 1461/2966] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:21:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 23 14:21:11.303716 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=59,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:21:17.472858 dv-cinder-825492-14-iscsi devstack@keystone.service[80395]: DEBUG dbcounter [-] [80395] Writing DB stats keystone:SELECT=5 {{(pid=80395) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 23 14:26:10.952769 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Authenticating user token {{(pid=80396) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 23 14:26:10.954219 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.fernet_utils [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80396) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 23 14:26:11.020302 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d34c853-afd0-42d6-88dc-89a8bb55e841 admin admin] Validating token access rules against request {{(pid=80396) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 23 14:26:11.026022 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe546103-6dbd-4180-869a-1285badf6523 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f40f1ccccf2f4b8e9c8373f984be3ddf', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '7e4bc8dfc8a242e2b66f47649e4dd59a', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'service', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80396) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 23 14:26:11.026686 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe546103-6dbd-4180-869a-1285badf6523 service placement] REQUEST_METHOD: `GET` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 23 14:26:11.026971 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe546103-6dbd-4180-869a-1285badf6523 service placement] SCRIPT_NAME: `/identity` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 23 14:26:11.027252 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe546103-6dbd-4180-869a-1285badf6523 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80396) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 23 14:26:11.036341 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe546103-6dbd-4180-869a-1285badf6523 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 23 14:26:11.038667 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe546103-6dbd-4180-869a-1285badf6523 service placement] RBAC: Authorization granted {{(pid=80396) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 23 14:26:11.048654 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: [pid: 80396|app: 0|req: 1506/2967] 10.222.0.70 () {60 vars in 1365 bytes} [Mon May 23 14:26:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 23 14:26:21.044329 dv-cinder-825492-14-iscsi devstack@keystone.service[80396]: DEBUG dbcounter [-] [80396] Writing DB stats keystone:SELECT=28 {{(pid=80396) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}